Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)

Meshy UltraTex Turns a Single Image into 4K PBR Textures Without Baked-In Highlights

Paste a photo of an object onto a 3D model as is, and the lamp's highlight sticks to the color and stays put when the lighting changes in the game. Meshy AI's UltraTex produces 4K PBR textures: unlit color, surface relief, metallic and roughness values each live in separate maps. The model first predicts color and normals, then metallic and roughness values, and six views are assembled into a single texture. There is no code yet, only a demo.

AI-processed from TG: @cgevent (Метаверсище и ИИще); edited by Hamidun News
Meshy UltraTex Turns a Single Image into 4K PBR Textures Without Baked-In Highlights
Source: TG: @cgevent (Метаверсище и ИИще). Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceTG: @cgevent (Метаверсище и ИИще) ↗
Editor-in-Chief
Key metric #1Normals are a map of surface micro-relief used to calculate lighting
Key metric #2Prompts from a vision-language model help identify the material
Key metric #3The code has not been released, and the model's license on Hugging Face is not specified

Imagine you've used a neural network to generate a metal chest for a game from a single photo. It looks great in the preview, but as soon as you rotate the lamp in the scene, the highlight stays where it was and the shadow from a fold doesn't move: both have been "baked" into the texture's color. On 18 November, Meshy AI published UltraTex on arXiv, a paper that tries to fix this. The system creates PBR textures (physically based materials, where color and light response are stored in separate maps) at 4K resolution from a single reference image, and tries to separate the material's true color from the shadows and highlights of the original lighting.

What's wrong with ordinary textures

If you take a photograph and project it onto a 3D model as is, the lighting under which it was shot comes along with the color. A bright spot from a lamp turns into "paint" on the surface. In a game engine or renderer where the lighting changes, such a model looks like a sticker.

PBR works differently. The material's color is stored separately, and other maps describe how the surface responds to light: normals (micro-relief), metallic and roughness. The highlight is then produced at render time and moves with the light source. The difficulty is that, from a single image, the system has to guess which part is true color and which is just a shadow or a reflection.

How UltraTex works

According to the authors, the system works in two stages. First, the model jointly predicts unlit color and surface normals for several views. Then a separate stage estimates metallic and roughness values.

To work out what an object is made of, the system uses prompts from a vision-language model. The model helps recognize the material: wood, metal, plastic, fabric. For example, if it sees polished metal, it is easier to conclude that a bright spot in the image is a reflection rather than light-colored paint.

Another technique concerns training. The model is trained on renders with controlled lighting, where the light is known in advance. This ties its predictions to the physical behavior of the surface, not just to how the image looks. In the described pipeline, maps from six views are assembled into 4K UV textures, that is, a ready-made set of unwrapped layouts that can be wrapped onto a 3D model.

The practical value of this decomposition shows when the scene lighting changes: the highlight should move with the light source rather than stay painted into the color texture. For environment artists, mobile game studios, and anyone making assets for AR and online stores, this is the difference between an image for a preview and an asset that can go straight into a scene.

What remains unclear and what to do about it

The work is new, and its limitations are obvious. There is a demo on Hugging Face, along with videos and a project page, but there is no code, and the model's license is not specified. That means you cannot currently integrate UltraTex into your own pipeline or fine-tune it on your own data, and commercial use of what is in the repository is legally questionable. The source materials contain no comparisons with competitors, so it is worth testing quality by hand in the demo.

Practical takeaways for anyone working with 3D and AI:

  • Evaluate generated assets under more than one lighting setup. Rotate the light source and check whether highlights and shadows stay "glued" to the texture.
  • When choosing a tool, check whether it outputs separate maps (unlit color, normals, metallic, roughness) rather than a single image.
  • If you are building a product around 3D generation, watch whether the code is released and which license Meshy chooses. That determines whether UltraTex becomes a working tool or remains a nice demo.

Meshy AI runs commercial 3D generation services, so similar ideas may appear in its products, but the source contains no confirmed plans. For now, UltraTex is a neat demonstration of a direction: from pretty pictures to materials that behave like real ones under any light.

Story

Key Questions Answered

What's wrong with ordinary textures?

If you take a photograph and project it onto a 3D model as is, the lighting under which it was shot comes along with the color. A bright spot from a lamp turns into "paint" on the surface. In a game engine or renderer where the lighting changes, such a model looks like a sticker. PBR works differently. The material's color is stored separately, and other maps describe how the surface responds to light: normals (micro-relief), metallic and roughness.

How UltraTex works?

According to the authors, the system works in two stages. First, the model jointly predicts unlit color and surface normals for several views. Then a separate stage estimates metallic and roughness values. To work out what an object is made of, the system uses prompts from a vision-language model. The model helps recognize the material: wood, metal, plastic, fabric. For example, if it sees polished metal, it is easier to conclude that a bright spot in the image is a reflection rather than light-colored paint.

What remains unclear and what to do about it?

The work is new, and its limitations are obvious. There is a demo on Hugging Face, along with videos and a project page, but there is no code, and the model's license is not specified. That means you cannot currently integrate UltraTex into your own pipeline or fine-tune it on your own data, and commercial use of what is in the repository is legally questionable. The source materials contain no comparisons with competitors, so it is worth testing quality by hand in the demo.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…