luckyrobots/open_phantom
13
0
Fork 0
mirror of https://github.com/luckyrobots/open_phantom.git synced 2025-04-04 02:52:18 +00:00
Code Issues 4 Packages Projects Releases Wiki Activity

update readme to switch mention of ml-depth-pro to MiDaS

Browse Source
This commit is contained in:
Ethan Clark 2025-03-24 15:17:14 -07:00
parent 8c26089951
commit 797527db9b
1 changed files with 3 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@ -8,7 +8,7 @@ Open Phantom is a fully open-source implementation of the approach described in
* **Camera-Only Data Collection** : Capture hand movements using any standard RGB camera * **Camera-Only Data Collection** : Capture hand movements using any standard RGB camera
* **3D Hand Tracking** : Convert 2D video to 3D hand poses using MediaPipe landmarks * **3D Hand Tracking** : Convert 2D video to 3D hand poses using MediaPipe landmarks
* **Advanced Depth Estimation** : Generate depth maps from monocular RGB input using ml-depth-pro * **Advanced Depth Estimation** : Generate depth maps from monocular RGB input using MiDaS
* **Hand Segmentation** : Precisely isolate hand regions with Meta's SAM2 for better depth estimation * **Hand Segmentation** : Precisely isolate hand regions with Meta's SAM2 for better depth estimation
* **ICP Registration** : Align hand mesh with depth point cloud for improved 3D accuracy * **ICP Registration** : Align hand mesh with depth point cloud for improved 3D accuracy
* **Anatomical Constraints** : Apply natural hand constraints to ensure realistic movements * **Anatomical Constraints** : Apply natural hand constraints to ensure realistic movements
@ -22,8 +22,8 @@ Open Phantom is a fully open-source implementation of the approach described in
Known limitations: Known limitations:
* ICP registration still being optimized for better alignment * In-painting human hand
* Depth estimation with ml-depth-pro is not implemented yet * Overlaying robot arm
* Performance optimizations needed for real-time processing * Performance optimizations needed for real-time processing
## Background ## Background
@ -60,11 +60,6 @@ cd external/sam2
pip install -e . pip install -e .
cd ../.. cd ../..
# Install dependencies for ML-Depth-Pro
cd external/ml-depth-pro
pip install -e .
cd ../..
# Verify installation # Verify installation
python -c "import mediapipe; import open3d; print('Dependencies successfully installed!')" python -c "import mediapipe; import open3d; print('Dependencies successfully installed!')"
``` ```
@ -152,7 +147,6 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
* This project is based on the research presented in "Phantom: Training Robots Without Robots Using Only Human Videos" * This project is based on the research presented in "Phantom: Training Robots Without Robots Using Only Human Videos"
* We use Meta's SAM2 (Segment Anything Model 2) for hand segmentation * We use Meta's SAM2 (Segment Anything Model 2) for hand segmentation
* ML-Depth-Pro from Apple provides advanced depth estimation
## Disclaimer ## Disclaimer