[osg-users] RayIntersector with KdTree causes a crash

Discussion:

Ale Maro

2018-07-26 09:30:17 UTC

Hi,

we have a problem with OSG 3.6.2 and Visual Studio 2017

We have a pick functionality in our software that uses RayIntersector.
We enabled KdTrees for this use.

When we apply the intersector to an object with a negative scale in the MatrixTransform parent node (e.g. scale = (-1, 1, 1)), it crashes inside the KdTree code.

If we disable kdtree everything works fine.
It also works with Visual Studio 2013 and OSG 3.4.0.

osg157-osgUtild.dll!std::vector<unsigned int,std::allocator<unsigned int> >::operator[](const unsigned __int64 _Pos) Line 1818 C++

osg157-osgUtild.dll!osg::KdTree::intersect<osg::TemplatePrimitiveFunctor<LineSegmentIntersectorUtils::IntersectFunctor<osg::Vec3d,double> > >(osg::TemplatePrimitiveFunctor<LineSegmentIntersectorUtils::IntersectFunctor<osg::Vec3d,double> > & functor, const osg::KdTree::KdNode & node) Line 160 C++
osg157-osgUtild.dll!osg::KdTree::intersect<osg::TemplatePrimitiveFunctor<LineSegmentIntersectorUtils::IntersectFunctor<osg::Vec3d,double> > >(osg::TemplatePrimitiveFunctor<LineSegmentIntersectorUtils::IntersectFunctor<osg::Vec3d,double> > & functor, const osg::KdTree::KdNode & node) Line 176 C++
osg157-osgUtild.dll!osg::KdTree::intersect<osg::TemplatePrimitiveFunctor<LineSegmentIntersectorUtils::IntersectFunctor<osg::Vec3d,double> > >(osg::TemplatePrimitiveFunctor<LineSegmentIntersectorUtils::IntersectFunctor<osg::Vec3d,double> > & functor, const osg::KdTree::KdNode & node) Line 176 C++
osg157-osgUtild.dll!osgUtil::LineSegmentIntersector::intersect(osgUtil::IntersectionVisitor & iv, osg::Drawable * drawable, const osg::Vec3d & s, const osg::Vec3d & e) Line 540 C++
osg157-osgUtild.dll!osgUtil::RayIntersector::intersect(osgUtil::IntersectionVisitor & iv, osg::Drawable * drawable) Line 122 C++
osg157-osgUtild.dll!osgUtil::IntersectionVisitor::intersect(osg::Drawable * drawable) Line 282 C++
osg157-osgUtild.dll!osgUtil::IntersectionVisitor::apply(osg::Geode & geode) Line 240 C++
osg157-osgd.dll!osg::Geode::accept(osg::NodeVisitor & nv) Line 37 C++
osg157-osgd.dll!osg::Group::traverse(osg::NodeVisitor & nv) Line 63 C++
osg157-osgFXd.dll!osgFX::Effect::inherited_traverse(osg::NodeVisitor & nv) Line 213 C++
osg157-osgFXd.dll!osgFX::Effect::traverse(osg::NodeVisitor & nv) Line 51 C++
osg157-osgd.dll!osg::NodeVisitor::traverse(osg::Node & node) Line 278 C++
osg157-osgUtild.dll!osgUtil::IntersectionVisitor::apply(osg::Group & group) Line 221 C++
osg157-osgFXd.dll!osgFX::Outline::accept(osg::NodeVisitor & nv) Line 54 C++
osg157-osgd.dll!osg::Group::traverse(osg::NodeVisitor & nv) Line 63 C++
osg157-osgd.dll!osg::NodeVisitor::traverse(osg::Node & node) Line 278 C++
osg157-osgUtild.dll!osgUtil::IntersectionVisitor::apply(osg::Transform & transform) Line 417 C++
osg157-osgd.dll!osg::NodeVisitor::apply(osg::MatrixTransform & node) Line 158 C++
osg157-osgd.dll!osg::MatrixTransform::accept(osg::NodeVisitor & nv) Line 37 C++
osg157-osgd.dll!osg::Group::traverse(osg::NodeVisitor & nv) Line 63 C++
osg157-osgd.dll!osg::NodeVisitor::traverse(osg::Node & node) Line 278 C++
osg157-osgUtild.dll!osgUtil::IntersectionVisitor::apply(osg::Group & group) Line 221 C++
osg157-osgd.dll!osg::Group::accept(osg::NodeVisitor & nv) Line 38 C++
osg157-osgd.dll!osg::Group::traverse(osg::NodeVisitor & nv) Line 63 C++
osg157-osgd.dll!osg::NodeVisitor::traverse(osg::Node & node) Line 278 C++
osg157-osgUtild.dll!osgUtil::IntersectionVisitor::apply(osg::Transform & transform) Line 417 C++
osg157-osgd.dll!osg::NodeVisitor::apply(osg::MatrixTransform & node) Line 158 C++
osg157-osgd.dll!osg::MatrixTransform::accept(osg::NodeVisitor & nv) Line 37 C++
osg157-osgd.dll!osg::Group::traverse(osg::NodeVisitor & nv) Line 63 C++
osg157-osgd.dll!osg::NodeVisitor::traverse(osg::Node & node) Line 278 C++
osg157-osgUtild.dll!osgUtil::IntersectionVisitor::apply(osg::Group & group) Line 221 C++
osg157-osgd.dll!osg::Group::accept(osg::NodeVisitor & nv) Line 38 C++
osg157-osgd.dll!osg::Group::traverse(osg::NodeVisitor & nv) Line 63 C++
osg157-osgShadowd.dll!osgShadow::ShadowedScene::traverse(osg::NodeVisitor & nv) Line 69 C++
osg157-osgd.dll!osg::NodeVisitor::traverse(osg::Node & node) Line 278 C++
osg157-osgUtild.dll!osgUtil::IntersectionVisitor::apply(osg::Group & group) Line 221 C++
osg157-osgShadowd.dll!osgShadow::ShadowedScene::accept(osg::NodeVisitor & nv) Line 36 C++
osg157-osgd.dll!osg::Group::traverse(osg::NodeVisitor & nv) Line 63 C++
osg157-osgd.dll!osg::NodeVisitor::traverse(osg::Node & node) Line 278 C++
osg157-osgUtild.dll!osgUtil::IntersectionVisitor::apply(osg::Group & group) Line 221 C++
osg157-osgd.dll!osg::NodeVisitor::apply(osg::ClipNode & node) Line 128 C++
osg157-osgd.dll!osg::ClipNode::accept(osg::NodeVisitor & nv) Line 35 C++
osg157-osgd.dll!osg::Group::traverse(osg::NodeVisitor & nv) Line 63 C++
osg157-osgd.dll!osg::NodeVisitor::traverse(osg::Node & node) Line 278 C++
osg157-osgUtild.dll!osgUtil::IntersectionVisitor::apply(osg::Group & group) Line 221 C++
osg157-osgd.dll!osg::Group::accept(osg::NodeVisitor & nv) Line 38 C++
osg157-osgd.dll!osg::Group::traverse(osg::NodeVisitor & nv) Line 63 C++
osg157-osgd.dll!osg::NodeVisitor::traverse(osg::Node & node) Line 278 C++
osg157-osgUtild.dll!osgUtil::IntersectionVisitor::apply(osg::Camera & camera) Line 498 C++
osg157-osgd.dll!osg::Camera::accept(osg::NodeVisitor & nv) Line 54 C++

Thank you in advance.

Berst regards

Ale

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74386#74386

Ale Maro

2018-07-26 09:33:11 UTC

Permalink

osg157-osgUtild.dll!std::vector<unsigned int,std::allocator<unsigned int> >::operator[](const unsigned __int64 _Pos) Line 1818 C++

Thank you in advance.

Berst regards

Ale

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74387#74387

Robert Osfield

2018-07-26 10:41:27 UTC

Permalink

Hi Ale,

The KdTree was refactored in 3.6.x to add support for points and lines
and PolytopeIntersector. While there has been lots of internal
changes in it, it of course shouldn't crash.

At this point in time I don't have any idea where the issue stems
from. Could you create a small example that reproduces the crash, I
or others can then step through what is happening and work out a fix.

Given this regression it looks like we'll have to make a 3.6.3 sooner
rather than later, but not before we fix the crash...

Cheers,
Robert.

Post by Ale Maro
Hi,
we have a problem with OSG 3.6.2 and Visual Studio 2017
We have a pick functionality in our software that uses RayIntersector.
We enabled KdTrees for this use.
When we apply the intersector to an object with a negative scale in the MatrixTransform parent node (e.g. scale = (-1, 1, 1)), it crashes inside the KdTree code.
If we disable kdtree everything works fine.
It also works with Visual Studio 2013 and OSG 3.4.0.

osg157-osgUtild.dll!std::vector<unsigned int,std::allocator<unsigned int> >::operator[](const unsigned __int64 _Pos) Line 1818 C++

Thank you in advance.
Berst regards
Ale
------------------
http://forum.openscenegraph.org/viewtopic.php?p=74387#74387
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

Ale Maro

2018-07-27 08:06:18 UTC

Permalink

Hi Robert,

my colleague reproduced the crash by modifying the osgPick example.
He created a sphere with a negative scaling transformation, build the kdtree and attached to the osgPick scene.
It crashes at the first intersection calculation.
If you do not build the kdtree or if the scaling transformation is positive it does not crash.

Attached is the modified osgPick.cpp

Thank you.

Best regards
Ale

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74398#74398

Attachments:
http://forum.openscenegraph.org//files/osgpick_163.cpp

Robert Osfield

2018-07-27 10:28:43 UTC

Permalink

Thanks for the test example. I don't have time right now to look into
it, but once I've achieved a couple of milestones on my Vulkan work
I'll return to clear up outstanding OSG bugs. Do I need any special
data to reproduce the crash?

Post by Ale Maro
Hi Robert,
my colleague reproduced the crash by modifying the osgPick example.
He created a sphere with a negative scaling transformation, build the kdtree and attached to the osgPick scene.
It crashes at the first intersection calculation.
If you do not build the kdtree or if the scaling transformation is positive it does not crash.
Attached is the modified osgPick.cpp
Thank you.
Best regards
Ale
------------------
http://forum.openscenegraph.org/viewtopic.php?p=74398#74398
http://forum.openscenegraph.org//files/osgpick_163.cpp
_______________________________________________
osg-users mailing list
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

Ale Maro

2018-07-27 12:04:51 UTC

Permalink

No special data needed.
The scene is built from code.

Cheers,
Ale

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=74400#74400