ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents

ScreenCoder is an intelligent UI-to-code generation tool that transforms any screenshot or design prototype into production-ready HTML/CSS code. It employs a modular multi-agent architecture that integrates visual understanding, layout planning, and adaptive code synthesis to generate accurate and editable front-end code.

ScreenCoder supports custom modifications, enabling developers and designers to adjust layouts and styles, and facilitates rapid prototyping and the creation of pixel-perfect interfaces.

How to Use ScreenCoder

1. Initial Generation with Placeholders: Run Python scripts to generate initial HTML code for a given screenshot.

   • Block Detection: Run the command python block_parsor.py.
   • Generation with Placeholders (Gray Image Blocks): Run the command python html_generator.py.

2. Final HTML Code: Run Python scripts to generate final HTML code with cropped images from the original screenshot.

   • Placeholder Detection: Run the command python image_box_detection.py.
   • UI Element Detection: Run the command python UIED/run_single.py.
   • Mapping Alignment Between Placeholders and UI Elements: Run the command python mapping.py.
   • Placeholder Replacement: Run the command python image_replacer.py.

3. Simple Execution: Run the command python main.py to generate the final HTML code.

Try ScreenCoder huggingface demo at Demo.