[osg-users] VAO Resource Leak in OSG 3.6.2

Discussion:

Daniel Emminizer, Code 5773

2018-08-02 14:29:10 UTC

Hi Robert,

I think I've found a resource leak in OSG 3.6.2. Attached is a contrived sample application that demonstrates the behavior.

What appears to be happening is that the VAO is being created with glGenVertexArrays(), but does not get deleted with glDeleteVertexArrays(). Using the open source API Trace, I can see that on exit it does destroy some VAO, but not (nearly) all of them.

I'm investigating now for a solution, but I'm not very familiar with this part of OSG. I'm passing this along now in case you see an obvious solution. Thanks,

- Dan

Robert Osfield

2018-08-03 08:11:29 UTC

Permalink

Hi Dan,

I'll be focusing on Vulkan work today so won't look into this right
away. Next week I'll set aside a day for OSG maintenance and
hopefully will be able to look at this then.

Robert.
On Thu, 2 Aug 2018 at 15:49, Daniel Emminizer, Code 5773

Post by Daniel Emminizer, Code 5773
Hi Robert,
I think I've found a resource leak in OSG 3.6.2. Attached is a contrived sample application that demonstrates the behavior.
What appears to be happening is that the VAO is being created with glGenVertexArrays(), but does not get deleted with glDeleteVertexArrays(). Using the open source API Trace, I can see that on exit it does destroy some VAO, but not (nearly) all of them.
I'm investigating now for a solution, but I'm not very familiar with this part of OSG. I'm passing this along now in case you see an obvious solution. Thanks,
- Dan
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

Daniel Emminizer, Code 5773

2018-08-03 10:13:49 UTC

Permalink

Hi Robert,

No problem. I'm out of "panic" mode since I found a work-around with the PR, so next week is just fine.

Thanks for the PR feedback. Unfortunately I'm unavailable all next week. I'm happy to retest any fix you have once I'm back.

I hope Vulkan work is going well. I'm watching the mailing list with anticipation.

- Dan

-----Original Message-----
Behalf Of Robert Osfield
Sent: Friday, August 03, 2018 4:11 AM
To: OpenSceneGraph Users
Subject: Re: [osg-users] VAO Resource Leak in OSG 3.6.2
Hi Dan,
I'll be focusing on Vulkan work today so won't look into this right
away. Next week I'll set aside a day for OSG maintenance and
hopefully will be able to look at this then.
Robert.
On Thu, 2 Aug 2018 at 15:49, Daniel Emminizer, Code 5773

Post by Daniel Emminizer, Code 5773
Hi Robert,
I think I've found a resource leak in OSG 3.6.2. Attached is a contrived

sample application that demonstrates the behavior.

Post by Daniel Emminizer, Code 5773
What appears to be happening is that the VAO is being created with

glGenVertexArrays(), but does not get deleted with glDeleteVertexArrays().
Using the open source API Trace, I can see that on exit it does destroy some
VAO, but not (nearly) all of them.

Post by Daniel Emminizer, Code 5773
I'm investigating now for a solution, but I'm not very familiar with this part

of OSG. I'm passing this along now in case you see an obvious solution.
Thanks,

Post by Daniel Emminizer, Code 5773
- Dan
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-

openscenegraph.org
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

Julien Valentin

2018-08-06 21:37:46 UTC

Permalink

A big old leak in memory too
Geometry destructor is never called for dynamically added!?

Post by Daniel Emminizer, Code 5773
Hi Robert,
No problem. I'm out of "panic" mode since I found a work-around with the PR, so next week is just fine.
Thanks for the PR feedback. Unfortunately I'm unavailable all next week. I'm happy to retest any fix you have once I'm back.
I hope Vulkan work is going well. I'm watching the mailing list with anticipation.
- Dan

-----Original Message-----
From: osg-users [mailto:] On
Behalf Of Robert Osfield
Sent: Friday, August 03, 2018 4:11 AM
To: OpenSceneGraph Users
Subject: Re: VAO Resource Leak in OSG 3.6.2
Hi Dan,
I'll be focusing on Vulkan work today so won't look into this right
away. Next week I'll set aside a day for OSG maintenance and
hopefully will be able to look at this then.
Robert.
On Thu, 2 Aug 2018 at 15:49, Daniel Emminizer, Code 5773

Post by Daniel Emminizer, Code 5773
Hi Robert,
I think I've found a resource leak in OSG 3.6.2. Attached is a contrived

sample application that demonstrates the behavior.

Post by Daniel Emminizer, Code 5773
What appears to be happening is that the VAO is being created with

glGenVertexArrays(), but does not get deleted with glDeleteVertexArrays().
Using the open source API Trace, I can see that on exit it does destroy some
VAO, but not (nearly) all of them.

Post by Daniel Emminizer, Code 5773
I'm investigating now for a solution, but I'm not very familiar with this part

of OSG. I'm passing this along now in case you see an obvious solution.
Thanks,

Post by Daniel Emminizer, Code 5773
- Dan
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-

openscenegraph.org
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
------------------
Post generated by Mail2Forum

------------------------
Twirling twirling twirling toward freedom

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74474#74474

Julien Valentin

2018-08-06 21:55:09 UTC

Permalink

Hi all
Oups wrote too fast, no memory leak sorry..:/

I fix the leak with

Code:
Geometry::~Geometry()
{
// do dirty here to keep the getGLObjectSizeHint() estimate on the ball
dirtyGLObjects();
//tag for delete associated
for(unsigned int i=0; i<_vertexArrayStateList.size(); ++i)
{
VertexArrayState* vas = _vertexArrayStateList[i].get();
if (vas) vas->release();
}
// no need to delete, all automatically handled by ref_ptr :-)
}

But don't know if it's the good place

Post by Julien Valentin
A big old leak in memory too
Geometry destructor is never called for dynamically added!?

Post by Daniel Emminizer, Code 5773
Hi Robert,
I think I've found a resource leak in OSG 3.6.2. Attached is a contrived

sample application that demonstrates the behavior.

Post by Daniel Emminizer, Code 5773
What appears to be happening is that the VAO is being created with

glGenVertexArrays(), but does not get deleted with glDeleteVertexArrays().
Using the open source API Trace, I can see that on exit it does destroy some
VAO, but not (nearly) all of them.

Post by Daniel Emminizer, Code 5773
I'm investigating now for a solution, but I'm not very familiar with this part

of OSG. I'm passing this along now in case you see an obvious solution.
Thanks,

Post by Daniel Emminizer, Code 5773
- Dan
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-

openscenegraph.org
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
------------------
Post generated by Mail2Forum

------------------------
Twirling twirling twirling toward freedom

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74475#74475

Guy Volckaert

2018-08-10 23:07:41 UTC

Permalink

I'm a little confused. Is there or is there not a memory leak?

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74494#74494

Julien Valentin

2018-08-11 00:08:34 UTC

Permalink

Hi
Sorry to create confusion.
There IS a GL object leak, VAO are created but not deleted
But there's no observable memory leak as GL objects are driver managed

Post by Guy Volckaert
I'm a little confused. Is there or is there not a memory leak?

------------------------
Twirling twirling twirling toward freedom

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74495#74495

Guy Volckaert

2018-08-11 00:27:12 UTC

Permalink

Got it. It's not necessarily a memory but a resource leak.

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74497#74497

Guy Volckaert

2018-08-11 00:53:34 UTC

Permalink

I suggest that we remove Geometry::dirtyGLObjects() as it does nothing other than call its base class, i.e. Drawable::dirtyGLObjects().

Code:

void Geometry::dirtyGLObjects()
{
Drawable::dirtyGLObjects();
}

I also noticed that the Drawable's destructor is calling a virtual function which is never a good idea, i.e. it's calling dirtyGLObjects(). I also suggest changing the dirtyGLObjects() to NOT BE virtual.

Code:

Drawable::~Drawable()
{
dirtyGLObjects(); // WARNING: Not a good idea to call a virtual function within a destructor.
}

Although Julien Valentin suggested changing the Geometry's destructor with respect resolving the Gl object resource leak, I wonder if it would not be better to move that code to the Drawable's destructor. Here is my suggestion instead:

Code:

void Drawable::~Drawable()
{
unsigned int i;
#ifdef OSG_GL_DISPLAYLISTS_AVAILABLE
for(i=0;i<_globjList.size();++i)
{
if (_globjList[i] != 0)
{
Drawable::deleteDisplayList(i,_globjList[i], getGLObjectSizeHint());
_globjList[i] = 0;
}
}
#endif

[color=red] for(i=0; i<_vertexArrayStateList.size(); ++i)
{
VertexArrayState* vas = _vertexArrayStateList[i].get();
if (vas) vas->release();
}[/color]
}

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74498#74498

Guy Volckaert

2018-08-11 01:06:40 UTC

Permalink

Just to clarify that my suggestion to change the Drawable's desructor is only if there is indeed a GL object resource leak, which I'm not totally convinced yet (by reviewing the code).

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74499#74499

Guy Volckaert

2018-08-13 16:14:22 UTC

Permalink

Julien,

Yo may want to verify if you have CACHE_NODES enabled. If so, then you may think there's a resource leak but in fact the scene graph leading to your Geometry may be cached by OSG.

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74511#74511

Daniel Emminizer, Code 5773

2018-08-13 16:23:18 UTC

Permalink

Hi Guy,

I attached to the original issue a use case that reproduces the problem. glDeleteVertexArrays() is never called on many of the arrays, although the associated geometry is destroyed. This goes until exit. The VAOs leak. This can be verified by running a profiler on GL and examining the OpenGL commands executed. The example does not use osgDB and wouldn't have CACHE_NODES enabled; please let me know if you think otherwise, I'd like to learn more.

Earlier someone mentioned not having a memory leak with this; but I am seeing on Windows and Linux associated and significant growth (over time) with these lost VAOs in Task Manager (Windows) and in top (Linux). So although it's technically a resource leak, that resource has (at least on my system) real memory associated with the running application.

I'm seeing this because real apps out there are encountering SIGKILL conditions under Linux due to running into the out-of-memory (OOM) handler. The code in the PR prevents this. Though as Julien points out, the PR could have issues with shared arrays amongst multiple geometries, but I don't have that use case in my apps and I do not see any examples in OSG that demonstrate this capability. We patched locally to prevent this SIGKILL in live code and have had no further reported issues in the last week and a half about it.

Hope this helps explain. Thanks,

- Dan

-----Original Message-----
Behalf Of Guy Volckaert
Sent: Monday, August 13, 2018 12:14 PM
Subject: Re: [osg-users] VAO Resource Leak in OSG 3.6.2
Julien,
Yo may want to verify if you have CACHE_NODES enabled. If so, then you
may think there's a resource leak but in fact the scene graph leading to your
Geometry may be cached by OSG.
------------------
http://forum.openscenegraph.org/viewtopic.php?p=74511#74511
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

Robert Osfield

2018-08-14 15:03:06 UTC

Permalink

Hi Dan,

I've just reached a mini milestone in my VSG Exploration work and
finished up my BOF presentation so I'm now kicking back a bit.
Tomorrow my plan is to dive back into the OSG maintenance side for a
day or two. In theory I have an hour left of work today, but feel
like relaxing more than thinking about bugs ;-)

Thanks for your patience,
Robert.

Robert Osfield

2018-08-16 17:33:09 UTC

Permalink

Hi Dan et al.

A quick update. I am now looking into this issue, sill too early to
provide a solution, it shouldn't take too much longer though.

Robert.

Daniel Emminizer, Code 5773

2018-08-16 17:39:27 UTC

Permalink

Hi Robert, thanks for the update. I'm happy to test my side against whatever you decide, once I'm in the office.

Thanks,

- Dan

-----Original Message-----
Behalf Of Robert Osfield
Sent: Thursday, August 16, 2018 1:33 PM
To: OpenSceneGraph Users
Subject: Re: [osg-users] VAO Resource Leak in OSG 3.6.2
Hi Dan et al.
A quick update. I am now looking into this issue, sill too early to
provide a solution, it shouldn't take too much longer though.
Robert.
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

Robert Osfield

2018-08-17 08:03:05 UTC

Permalink

Hi Dan,

On Fri, 17 Aug 2018 at 07:04, Daniel Emminizer, Code 5773

Post by Daniel Emminizer, Code 5773
Hi Robert, thanks for the update. I'm happy to test my side against whatever you decide, once I'm in the office.

I'm happy with what I've checked in, so now it's just a matter of wider testing.

Cheers,
Robert.

Daniel Emminizer, Code 5773

2018-08-20 12:39:21 UTC

Permalink

Hi Robert,

Just got finished testing on my side. In the full application I'm seeing no problems, looks good all around. I agree all closed out. Thanks for your help.

- Dan

-----Original Message-----
Behalf Of Robert Osfield
Sent: Friday, August 17, 2018 4:03 AM
To: OpenSceneGraph Users
Subject: Re: [osg-users] VAO Resource Leak in OSG 3.6.2
Hi Dan,
On Fri, 17 Aug 2018 at 07:04, Daniel Emminizer, Code 5773

Post by Daniel Emminizer, Code 5773
Hi Robert, thanks for the update. I'm happy to test my side against

whatever you decide, once I'm in the office.
I'm happy with what I've checked in, so now it's just a matter of wider testing.
Cheers,
Robert.
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

Robert Osfield

2018-08-16 18:50:33 UTC

Permalink

I have checked in a fix for the leaking of the VAS objects:

https://github.com/openscenegraph/OpenSceneGraph/commit/eee5d5482e41290e7c210273f8c6bae489cedb76

The solution was to call the releaseGLObjects(), but done in a way to
avoid a the Drawable destructor calling the
Geometry::releaseGLObjects() - I explictly call the
Drawable::releaseGLObjects() and Geometry::releaseGLObjects() within
the respective destructors. I also modified the releaseGLObjects() to
properly handle the release of the VAS objects.

With this fix the leaking_vao test program no longer consumes and ever
increasing amount of memory :-)

The fix is checked into OSG-3.6 and master.

Cheers,
Robert.