Meta Description: How I built a full-stack image-to-3D pipeline using Gemini, Hunyuan3D-2, LangGraph, FastAPI, Trimesh, and WebGL. I wanted to experiment with a problem that sounds simple: Can a single 2D image be turned into a usable 3D model? A single image gives us information about the visible surface, but says very little about the hidden geometry. The challenge isn't just generating a mesh; it is handling the entire pipeline around it. So I built Core3D, a system that takes a single image, preprocesses it, adds geometric context, generates a 3D mesh, repairs and validates the geometry, and finally lets the user inspect the resulting .GLB model directly in the browser. The end-to-end pipeline averages roughly 22 seconds in the project's benchmark. Architecture The application is split between a Next.js frontend and a FastAPI backend. ┌──────────────────────┐ │ Next.js Frontend │ │ WebGL Studio │ └──────────┬───────────┘ │ HTTP / GLB │ ┌──────────▼───────────┐ │ FastAPI API │ └──────────┬───────────┘ │ LangGraph Pipeline │ ┌──────────────────┼──────────────────┐ │ │ │ ▼ ▼ ▼ Preprocessing VLM Conditioning 3D Generation U2-Net Gemini Hunyuan3D-2 │ │ │ └──────────────────┼──────────────────┘ ▼ Mesh Processing Trimesh │ ▼ Final GLB The frontend uses Next.js, React, TypeScript, Tailwind CSS, and Google's <model-viewer>. The backend uses FastAPI, LangGraph, LangChain, and Pydantic. 1. Preprocessing the Image The first problem is background noise. If the input image contains a table, wall, floor, or other objects, the 3D reconstruction model has to distinguish the actual subject from everything else. I use rembg with U2-Net to isolate the foreground object. Input Image ↓ U2-Net / rembg ↓ Foreground Mask ↓ Bounding Box ↓ Center + Pad ↓ 1024 × 1024 RGBA After segmentation, the subject is cropped from its bounding box, scaled with approximately 15% padding, centered, and resized to 1024 × 1024. This gives the reconstruction model a much more consistent input regardless of how the original photograph was framed. 2. The Problem With a Single View A single image only exposes one side of an object. For example, a photograph of a chair might show the seat, front legs, and backrest while giving us no direct information about the rear structure. The model therefore has to infer: Hidden surfaces Object thickness Rear geometry Structural connections Overall volume Instead of relying only on the image, I added a multimodal conditioning step. 3. Using Gemini as a Geometric Conditioning Layer The preprocessed image is sent to Gemini 3.1 Flash-Lite. Gemini isn't responsible for generating the mesh. Instead, it produces a compact description of the object's 3D structure, including its volume, materials, shape, and likely hidden geometry. Image │ ▼ Gemini Vision Model │ ▼ Geometric Description │ ▼ 3D Reconstruction This makes the vision-language model act as a conditioning layer for the 3D model. For example, instead of giving the reconstruction model only: "chair" it can receive a more useful description such as: "Minimalist wooden chair with a rectangular seat, four tapered legs, solid rear support and smooth wooden surfaces." The implementation also has a fallback path so that if the multimodal request fails, the pipeline can continue using the user's prompt or a default description. 4. Generating the 3D Shape Once the image has been prepared and conditioned, it is sent to Hunyuan3D-2 through a Hugging Face ZeroGPU Space. The backend uses gradio_client to communicate with the remote inference service. The current configuration includes: Steps: 30 Guidance Scale: 5.5 Octree Resolution: 256 Num Chunks: 8000 The generated result is returned as a .GLB file. The interesting part here is that the expensive 3D inference doesn't have to run directly inside the FastAPI process. FastAPI │ ▼ Hugging Face ZeroGPU │ ▼ Hunyuan3D-2 │ ▼ Raw GLB This keeps the application backend relatively lightweight while still exposing the model through a normal API. 5. Why the Raw Mesh Isn't the Final Mesh A generated mesh isn't necessarily ready for rendering. It can contain: Incorrect normals Inverted faces Small holes Unnecessary geometry Excessive polygon counts So the generated GLB is passed through a separate mesh-processing stage using Trimesh. Raw GLB ↓ Load Geometry ↓ Fix Inversions ↓ Fix Normals ↓ Fill Holes ↓ Decimate if Necessary ↓ Validate ↓ Final GLB The pipeline uses Trimesh repair functions to fix inversions, normals, and holes. It also applies polygon reduction when the mesh exceeds 45,000 faces. This matters because a high-polygon model isn't necessarily a better model for an interactive browser application. 6. Validation and Retry Logic I didn't want to blindly trust whatever geometry the model generated. The pipeline tracks information such as: face_count is_valid retry_count max_retries After mesh processing, LangGraph determines whether the result is acceptable. ┌──────────────┐ │ Mesh Repair │ └──────┬───────┘ │ ▼ Is Geometry Valid? / \ Yes No │ │ ▼ ▼ END Retry │ ▼ 3D Generation The current configuration allows one retry when validation fails. This is one reason LangGraph fits the project well: the workflow isn't just a straight sequence of function calls. It can make decisions and route execution accordingly. 7. LangGraph as the Orchestration Layer The entire backend pipeline is represented as a state machine. The shared state contains fields such as: class MeshState(TypedDict): job_id: str input_image_path: str preprocessed_path: Optional[str] user_prompt: Optional[str] enhanced_prompt: Optional[str] raw_mesh_path: Optional[str] final_mesh_path: Optional[str] face_count: int is_valid: bool retry_count: int max_retries: int The resulting workflow looks like: START ↓ Preprocess ↓ Generate Context ↓ 3D Inference ↓ Mesh Repair ↓ Validation ├── Valid → END └── Invalid → Retry Each stage reads from and updates the same graph state. That makes the system much easier to extend than a single large backend function. 8. FastAPI as the Model Gateway The frontend communicates with the backend through a small API surface. The main generation endpoint is: POST /generate Content-Type: multipart/form-data The request contains: image: binary file prompt: optional string The backend creates a job ID, stores the uploaded image, initializes the graph state, and executes the LangGraph pipeline. When processing finishes, the final .GLB is returned. Additional endpoints provide model retrieval and health checking: GET /models/{filename} GET /health This keeps the frontend independent from the individual AI services involved in reconstruction. 9. Building a Browser-Based 3D Studio Generating a 3D model isn't enough. You also need a way to inspect it. The /inspect page uses Google's <model-viewer> component to render the generated GLB directly in the browser. The user can: Rotate the model Zoom Enable auto-rotation Change color Adjust roughness Adjust metallic properties Modify exposure Change transparency Material properties are updated directly through the model-viewer's PBR material interface. Generated GLB ↓ <model-viewer> ↓ WebGL Renderer ↓ Interactive 3D Model This turns the output from a generated file into something the user can immediately inspect. 10. Browser-Side Model Persistence Large 3D files don't need to be downloaded repeatedly. The frontend stores generated GLB assets in IndexedDB. Generated GLB ↓ IndexedDB ↓ Browser Reload ↓ Restore Local Blob ↓ Render Model The application keeps the generated binary locally so that the inspection page can restore the asset after a reload without regenerating it. 11. Performance The benchmark in the project shows approximately: Stage Average Time U2-Net Alpha Matting ~1.2 s Gemini Conditioning ~1.1 s Hunyuan3D-2 ~18.5 s Mesh Repair ~1.4 s Total ~22.2 s The numbers make one thing very clear: 3D generation is the bottleneck. Almost all of the latency comes from the Hunyuan3D inference stage. That means future optimization efforts would have a much larger impact if focused on the reconstruction model rather than the preprocessing pipeline. Tech Stack Layer Technology Frontend Next.js, React, TypeScript Styling Tailwind CSS 3D Rendering Google <model-viewer> / WebGL Backend FastAPI Orchestration LangGraph Vision / Conditioning Gemini 3.1 Flash-Lite 3D Generation Hunyuan3D-2 Segmentation U2-Net / rembg Mesh Processing Trimesh Storage IndexedDB Remote Inference Hugging Face ZeroGPU What I Found Most Interesting The interesting part of an image-to-3D application isn't really the phrase "image to 3D." There are several different problems hiding behind it: 2D Image ↓ Foreground Isolation ↓ Spatial Normalization ↓ Geometric Conditioning ↓ 3D Reconstruction ↓ Mesh Repair ↓ Validation ↓ WebGL Rendering Each stage solves a different problem. The segmentation model removes irrelevant background information. The vision-language model adds semantic information about geometry. Hunyuan3D generates the actual mesh. Trimesh makes that geometry more usable. LangGraph coordinates the entire workflow. And WebGL turns the result into something interactive. What's Next? The biggest improvement I'd like to explore is multi-view reconstruction. A single image inherently limits what the model can know. Providing multiple views should reduce ambiguity around hidden geometry and improve structural accuracy. Other improvements could include: Local GPU inference Texture generation Better mesh optimization Automatic quality scoring Multi-image input Persistent model history Additional export formats Conclusion Core3D started with a simple question: Can a 2D image become a 3D model? The interesting part was discovering how many systems are required to make that actually useful. The final pipeline combines computer vision, multimodal reasoning, generative 3D reconstruction, geometry processing, stateful orchestration, API design, and browser-based WebGL. Single Image ↓ U2-Net / rembg ↓ 1024 × 1024 Subject ↓ Gemini Geometric Context ↓ Hunyuan3D-2 ↓ Raw GLB ↓ Trimesh Repair ↓ Validation / Retry ↓ Interactive WebGL ↓ Final 3D Asset The biggest takeaway for me was that generative AI becomes much more useful when it is treated as one component inside a larger engineering system. The model generates the geometry. The rest of the system makes that geometry usable. Source Code GitHub: https://github.com/iPrq/image-to-model