Gemini and semantic search: AI matches furniture to blueprints with 87% accuracy

Selecting furniture based on an architectural drawing is a task that in construction and interior design is still largely done manually. A specialist takes a drawing, opens a supplier's catalog, and methodically goes through positions: comparing dimensions, materials, style. One project takes 20 to 40 person-hours just on this stage.

When there are multiple objects, costs multiply proportionally. A Russian team of developers set out to close this gap by building an AI system that automates the entire selection process and delivers recommendations with 87% accuracy. The problem lies at the intersection of computer vision and information retrieval.

An architectural drawing is a specialized document with notations, conventional symbols, scale grids, and multilayered layers. Simply recognizing an object in an image is not enough: you need to understand the planning solution as a whole, identify functional zones, extract seating areas for specific furniture categories, and account for real space constraints. The heart of the system is a multimodal architecture with Gemini as the main coordinator.

This model takes on understanding the drawing: recognizes the layout, identifies rooms, determines where the bedroom is, where the living room is, what are the circulation zones and where there are constraints on height, lighting, or wall configuration. This is not just OCR and not trivial object recognition — Gemini works with the semantics of the architectural document, extracting structured data for the next layer of the system. After the drawing is broken down into semantic units, semantic search across the catalog comes into play.

Each item has already undergone vectorization: product characteristics — dimensions, material, style, color palette, price segment — are transformed into vector embeddings. The system matches the requirements from the drawing with this vector representation and finds the closest matches. The output is not just a list of hundreds of suitable items, but a ranked selection with an explanation of why this particular model is recommended for a specific place in the plan.

The technical architecture is not limited to two components. In addition to Gemini and semantic search, the pipeline involves models for pre-processing drawings: scale normalization, layer separation, cleaning of scanning artifacts. Real drawings from design organizations come in different states — from clean CAD exports to scanned paper documents with tears and stains.

The system must work stably with this entire range without manual preprocessing. Achieving 87% accuracy was not achieved on the first try. The team iterated over several key nodes: the quality of parsing drawings in different formats, the strategy for vectorizing catalog data, and the final ranking mechanism.

A particular challenge was unusual layouts — when an architect uses non-standard notations or the drawing contains only a fragment of the room. For edge cases, they added fallback logic and an additional validation layer for results. The practical value of the development is in scaling specialist working time.

If previously a designer could thoroughly work out 2–3 projects per week, with the AI system they check and correct already ready recommendations instead of formulating them from scratch. For construction companies working with standard residential complexes, this means the ability to manage dozens of objects in parallel without a proportional increase in staff. The project demonstrates how multimodal AI systems are beginning to automate operational tasks that were long considered too specialized for machine processing.

Architectural drawings are a complex type of unstructured data, and the fact that Gemini handles their semantic parsing opens the door for similar solutions in related fields: engineering schematics, structural drawings, technical specifications. The next logical step is integration with BIM platforms and direct export of recommendations to project documentation.