convert 3d to 2d

[hasanshamsijeze]

How can I convert a 3D position into a 2D screen position so that I can determine which (x, y) screen coordinates correspond to a 3D object with (x, y, z) coordinates, and check whether that object overlaps with the mouse position?

[gwaldron]

Generally speaking, you would multiply the 3D point by the modelview and projection matrices and then scale that value to the viewport size.

pseudocode:

clip = projectionMatrix * viewMatrix * point3D;
ndc = (clip.xy + 1.0) * 0.5;
screen = viewport.xy + (ndc * viewport.size);

You can find the matrices and viewport in the vsg::Camera object.

[hasanshamsijeze]

thanks this worked

[derkreature]

Following up on gwaldron.
There are a few ways you can perform your picking too.
This is very much example code, but I thought it wouldn't hurt to share.
In the case of Rocky, I'd use doubles for math.

1.) CPU version. This can also be implemented using Cuda or Compute Shaders but takes some work to play nice on the GPU. This is somewhat inefficient if you don't have a spatial acceleration structure (something to cull unrequired objects, say a frustum test around a bounding primitive type):
Convert your ray into object space.
This becomes something like (You will probably need to transpose or adjust due to my conventions here:
`
// Screenspace Ray
Vector3f v;
v.x = ( ( ( 2.0f * screenPosition.x ) / width ) - 1 ) / pmatProj._11;
v.y = -( ( ( 2.0f * screenPosition.y ) / height ) - 1 ) / pmatProj._22;
v.z = 1.0f;

// View matrix
Matrix44f view = camera->viewMatrix();

// For each object concatenate view * world, inverse view to world (I'd add a test around this if you are in screenspace, there are 
// other use cases where you may want to use something like embree to scatter millions of rays over a bvh or some other structure
// though below is reasonably fine, you could even run a parallel for loop over this. 
for each (mesh in meshCache)
{
       Matrix44f world = mesh->worldTransform();
       Matrix44f worldView = view * world;
       Matrix44f inverseWorldView = worldView.inverse();
        
        // Screen space pick ray to object space
        ray.dir.x  = inverseWorldView.x*m._11 + inverseWorldView.y*m._21 + inverseWorldView.z*m._31;
        ray.dir.y  = inverseWorldView.x*m._12 + inverseWorldView.y*m._22 + inverseWorldView.z*m._32;
        ray.dir.z  = inverseWorldView.x*m._13 + inverseWorldView.y*m._23 + inverseWorldView.z*m._33;
        ray.origin.x = m._41;
        ray.origin.y = m._42;
        ray.origin.z = m._43;    
        
        // Now you can use this ray to test against triangles            
        auto result = rayIntersect(ray, mesh);
        
        // or test against capsules
        auto result = rayIntersectCollisionPrimitive(ray, mesh->collisionHull()); 
   
        // Result is usually something like this, should be easy enough for you to wire up a few collision shapes or triangle ray intersection.            
        struct TriangleSelectionIntersection
        {
            size_t  object_id;

            // mesh face that was intersected
            uint32_t face;                     
            
            // barycentric coords of intersection
            Vector2f barycentric; 
            
            // distance from ray origin to intersection
            float dist;                   
        
            // Position of intersection.
            Vector3f position;
        
            // Face normal at position
            Vector3f normal;
        };
}

// Meah, for completeness I'll throw this in too. Box3f is basically what Imath uses.
// This version assumes object space, you could also supply a transform to get to world space and test rays in world.
inline bool
intersectAABB(const Box3f& aabb,
                        const Ray& r)
{
    
    const Vector3f& newMin = aabb.min;
    const Vector3f& newMax = aabb.max;

    Vector3f lb, rt;

    lb.x = std::min(newMax.x, newMin.x);
    lb.y = std::min(newMax.y, newMin.y);
    lb.z = std::min(newMax.z, newMin.z);

    rt.x = std::max(newMax.x, newMin.x);
    rt.y = std::max(newMax.y, newMin.y);
    rt.z = std::max(newMax.z, newMin.z);

    // r.dir is unit direction vector of ray
    Vector3f dirfrac;
    dirfrac.x = 1.0f / r._direction.x;
    dirfrac.y = 1.0f / r._direction.y;
    dirfrac.z = 1.0f / r._direction.z;

    // lb is the corner of AABB with minimal coordinates - left bottom, rt is maximal corner
    // r.org is origin of ray
    float t1 = (lb.x - r._origin.x)*dirfrac.x;
    float t2 = (rt.x - r._origin.x)*dirfrac.x;
    float t3 = (lb.y - r._origin.y)*dirfrac.y;
    float t4 = (rt.y - r._origin.y)*dirfrac.y;
    float t5 = (lb.z - r._origin.z)*dirfrac.z;
    float t6 = (rt.z - r._origin.z)*dirfrac.z;

    float tmin = std::max(std::max(std::min(t1, t2), std::min(t3, t4)), std::min(t5, t6));
    float tmax = std::min(std::min(std::max(t1, t2), std::max(t3, t4)), std::max(t5, t6));

    // if tmax < 0, ray (line) is intersecting AABB, but whole AABB is behind
    if (tmax < 0)
    {
        return false;
    }
    // if tmin > tmax, ray doesn't intersect AABB
    else if (tmin > tmax)
    {
        return false;
    }
    return true;
}

inline bool 
intersectTriangle (const Vector3f& orig, const Vector3f& dir, 
                            const Vector3f& v0, const Vector3f& v1, const Vector3f& v2, 
                            float& t, float& u, float& v )
{

    // Find vectors for two edges sharing vert0
    Vector3f edge1 = v1 - v0;
    Vector3f edge2 = v2 - v0;

    // Begin calculating determinant - also used to calculate U parameter
    Vector3f pvec = dir.cross(edge2);

    // If determinant is near zero, ray lies in plane of triangle
    float det = edge1.dot(pvec);

    Vector3f tvec;
    if( det > 0 )
    {
        tvec = orig - v0;
    }
    else
    {
        tvec = v0 - orig;
        det = -det;
    }

    if( det < 0.0001f )
        return false;

    // Calculate U parameter and test bounds
    u = tvec.dot(pvec);
    if (u < 0.0f || u > det )
        return false;

    // Prepare to test V parameter
    Vector3f qvec = tvec.cross(edge1 );

    // Calculate V parameter and test bounds
    v = dir.dot(qvec);
    if (v < 0.0f || u + v > det )
        return false;

    // Calculate t, scale parameters, ray intersects triangle
    t = edge2.dot(qvec );
    float invDet = 1.0f / det;
    t *= invDet;
    u *= invDet;
    v *= invDet;

    return true;
}
}        

`

2.) Is hardware selection using rasterization. Bgfx has a good starter example. This is very useful if you need to sample many picks or you need simple picks against skinned or deformed geometry.
https://github.com/bkaradzic/bgfx/tree/master/examples/30-picking

1.) Rasterize an ID buffer during your GBuffer or surface shading pass. For anything blended you will need to rasterize the ID buffer after surface. (You could also just rasterize the collision primitives too).
2.) During a pick pass, resolve the depth buffer and transform the depth (and denorm depth) back to world or local using inverse of proj * view * world or proj * view. 
3.) Sample ID buffer at pick coord.
4.) Read back result/s async.

3.) Compute or using hardware raytracing.
A bit beyond the purpose of my reply, but if you are simulating something very large and don't have a viewpoint, this is the way I go.