Infinite Synthetic Data Generation for Embodied AI

Overview

Lucky World is a state-of-the-art simulator built by Lucky Robots to create hyperrealistic environment interactions using the power of Unreal Engine 5 and MuJoCo physics. Designed from the ground up for robotic learning, Lucky World enables you to generate infinitely many hyperrealistic demonstrations for training robust robotic systems that can easily bridge the gap the sim2real gap.

Core Technology

• Unreal Engine 5 - Leveraging Nanite virtualized micropolygon geometry and Lumen global illumination for immaculate visual fidelity
• MuJoCo Physics - Industry-leading physics simulation ensuring accurate object interactions, contacts, and material properties
• Procedural Generation - Dynamic environment creation with randomized object placement, lighting conditions, and material properties
• Sensor Simulation - High-fidelity simulated sensors including RGB cameras, depth sensors, LiDAR, and IMUs

Features

• Hyperrealistic Environments - Meticulously crafted indoor scenes with physically based materials and lighting
• Dynamic Interactions - Fully interactive objects with realistic physics responses and material properties
• Advanced Robotics - Pre-configured robots including Stretch v1 and Franka Arm with accurate kinematic models
• Configurable Difficulty - Adjustable environment complexity, lighting conditions, and object arrangements
• Data Collection - Built-in tools for capturing ground truth data, including depth maps, segmentation masks, and bounding boxes
• Real-time Rendering - Optimized for real-time performance on modern hardware

System Requirements

• Operating Systems: Windows 10/11, macOS, Linux (Ubuntu 20.04+)
• Hardware:
  • GPU: NVIDIA GTX 1080 or AMD equivalent (minimum), RTX 3080 or equivalent (recommended)
  • CPU: 8-core processor (recommended)
  • RAM: 16GB (minimum), 32GB (recommended)
  • Storage: 50GB available SSD space
• Software:
  • Unreal Engine 5.5
  • Git LFS
  • Visual Studio 2022 or Rider (Windows)
  • Xcode 14+ (macOS)
  • Clang or GCC (Linux)

Installation

Follow these steps to pull down the source code and compile the project:

1. Clone the Repository

Important: Don't download the .zip file. It's best to use the CLI or a visual tool so Git LFS can install the proper libraries.

git clone https://luckyrobots.com/luckyrobots/luckyworld.git
cd luckyworld

2. Install Git LFS

Install Git LFS based on your operating system:

macOS:

brew install git-lfs

Linux:

sudo yum install git-lfs

Windows:

# Install via Visual Installer
# https://git-scm.com/download/win
# (Be sure to select the option to install git-lfs)

# Or via Chocolatey
choco install git

3. Pull LFS Files

Run this command to ensure all large binary files are properly downloaded:

git lfs pull

4. Setup Project Files

Windows:

• Right-click the .uproject file
• Select "Generate Visual Studio Project Files"
• Open the generated .sln file with your IDE (i.e. Visual Studio or Rider)

macOS/Linux:

• Right-click the .uproject file
• Follow platform-specific steps to generate project files

5. Engine Version Note

The project starts in UE 5.5 binary version. If you're using source:

• Right-click the .uproject file
• Select "Switch Unreal Engine version..."
• Choose your installed engine version

6. Build and Run

• Debug 'luckyrobots'

Using Lucky World

Editor Mode

Lucky World can be used directly in the Unreal Editor for:

1. Environment Creation:

   • Design custom environments using UE5's powerful editor tools
   • Place and configure objects, furniture, and obstacles
   • Adjust lighting, materials, and atmospheric conditions

2. Robot Configuration:

   • Configure robot parameters through the Robot Blueprint editor
   • Adjust sensor placement, field of view, and resolution
   • Calibrate physics parameters for realistic robot movement

3. Simulation Settings:

   • Set simulation frame rate and physics substeps
   • Configure environment difficulty and randomization parameters
   • Adjust camera properties and sensor noise levels

Standalone Mode

For pure simulation or data collection:

1. Package the project (File > Package Project)
2. Run the standalone executable
3. Configure simulation parameters via the settings menu
4. Collect data using the built-in recording tools

Configuration

Environments

Lucky World includes several pre-built environments:

• Loft
• Rome
• Paris
• Marseille
• Istanbul
• Kitchen (for Panda Arm)

Each environment can be customized through the Content Browser (Content/Environments/).

Troubleshooting

Common Issues

Missing Assets / LFS Files:

git lfs install
git lfs pull

Compilation Errors:

• Verify correct Unreal Engine version is selected
• Clear derived data folder ([Engine]/DerivedDataCache/)
• Rebuild project

Performance Issues:

• Adjust rendering settings in Edit > Project Settings > Engine > Rendering
• Lower simulation physics detail in Edit > Project Settings > Engine > Physics

Crashes on Startup:

• Check logs in [ProjectFolder]/Saved/Logs/
• Verify GPU drivers are up to date

License

Lucky World is licensed under the Polyform License - see the LICENSE file for details.