启动 HN：Flywheel（YC S25）——挖掘机的 Waymo
大家好，
我是Jash，Flywheel AI的联合创始人（<a href="https://useflywheel.ai">https://useflywheel.ai</a>），之前曾在<a href="https://remoteteleop.com" rel="nofollow">https://remoteteleop.com</a>工作。我的联合创始人Mahimana Bhatt曾在自动驾驶公司Motional负责机器学习数据和评估管道的构建。
我们正在为挖掘机构建远程遥控和自动化系统。
与现有挖掘机进行接口以实现远程遥控（或自动化）是非常困难的。与使用线控技术的汽车不同，数百万台挖掘机大多数是全液压机器。操纵杆连接到一个主液压回路中的控制液压电路，按比例移动主液压回路中的缸体，最终移动挖掘机的关节。这意味着挖掘机通常没有电子元件来控制关节。我们通过机械方式驱动挖掘机内部的操纵杆和踏板来解决这个问题。
我们通过改装的方式，使其适用于任何型号和品牌的挖掘机，从而增强现有机器的功能。通过实现远程遥控，我们能够提高现场安全性、生产力以及成本效率。
操作员的（远程）遥控使我们能够准备自动化所需的训练数据。在机器人技术中，训练数据包括观察和动作。虽然互联网上有大量的图像和视频，但以自我为中心的（视角）观察和动作数据极为稀缺，正是这种稀缺性阻碍了机器人学习策略的扩展。
Flywheel通过准备来自我们已经部署的远程遥控挖掘机的训练数据来解决这个问题。我们只需非常少量的硬件设置和资源即可实现。
在我们参加YC期间，我们进行了25到30次传感器堆栈和位置排列/组合的迭代，以及模型超参数的变化。我们称之为“我们改装物理形态的演变”。最终，我们达到了当前的演变，并成功地在仅有几个小时的训练数据下训练出了一定程度的自动化。我们将继续调整这些“旋钮”，以找出更有效的方法。
最大的收获是数据的重要性远超模型超参数的优化。因此，今天我们将开源使用Flywheel系统在真实施工现场收集的100小时挖掘机数据集。这是与Frodobots.ai合作的成果。
一些详细信息如下：
<a href="https://youtu.be/zCNmNm3lQGk" rel="nofollow">https://youtu.be/zCNmNm3lQGk</a>
机器/改装的详细信息：
沃尔沃EC380（38吨挖掘机）
4个摄像头（25fps）
25Hz专家操作员的动作数据
我们刚刚起步。我们在光照、天气、任务等方面有大量变化，并且很快会增加更多小时的数据，并转换为lerobot格式。我们这样做是为了让像你我这样的开发者能够在真实世界数据上尝试训练模型，而这些数据是非常难以获取的。
请查看数据集，随意下载和使用。我们迫不及待想看到你们的成果。我会在讨论中保持在线，期待社区的评论和反馈！
数据集链接：<a href="https://huggingface.co/datasets/FlywheelAI/excavator-dataset" rel="nofollow">https://huggingface.co/datasets/FlywheelAI/excavator-dataset</a>
查看原文
Hey HN,
I’m Jash, cofounder of Flywheel AI (<a href="https://useflywheel.ai">https://useflywheel.ai</a>) and formerly <a href="https://remoteteleop.com" rel="nofollow">https://remoteteleop.com</a>. My cofounder, Mahimana Bhatt, built ML data and eval pipeline at autonomous driving company Motional.
We’re building remote teleop and autonomous stack for excavators.<p>Interfacing with existing excavators for enabling remote teleop (or autonomy) is hard. Unlike cars which use drive-by-wire technology, most of the millions of excavators are fully hydraulic machines. The joysticks are connected to a pilot hydraulic circuit, which proportionally moves the cylinders in the main hydraulic circuit which ultimately moves the excavator joints. This means excavators mostly do not have an electronic component to control the joints. We solve this by mechanically actuating the joysticks and pedals inside the excavators.<p>We do this with retrofits which work on any excavator model/make, enabling us to augment existing machines. By enabling remote teleoperation, we are able to increase site safety, productivity and also cost efficiency.<p>(Remote) Teleoperation by the operators enables us to prepare training data for autonomy. In robotics, training data comprises observation and action. While images and videos are abundant on the internet, egocentric (pov) observation and action data is extremely scarce, and it is this scarcity that is holding back scaling robot learning policies.<p>Flywheel solves this by preparing the training data coming from our remote teleop-enabled excavators which we have already deployed. And we do this with very minimal hardware setup and resources.<p>During our time in YC, we did 25-30 iterations of sensor stack and placement permutations/combinations, and model hyperparams variations. We called this “evolution of the physical form of our retrofit”. Eventually, we landed to our current evolution and have successfully been able to train some levels of autonomy with only a few hours of training data. We will continue playing around with these ‘knobs’ to figure out what works better.<p>The big takeaway was how much more important data is than optimizing hyperparams of the model. So today, we’re open sourcing 100hrs of excavator dataset that we collected using Flywheel systems on real construction sites. This is in partnership with Frodobots.ai<p>Some details here:
<a href="https://youtu.be/zCNmNm3lQGk" rel="nofollow">https://youtu.be/zCNmNm3lQGk</a><p>Machine/retrofit details:
Volvo EC380 (38 ton excavator)
4xcamera (25fps)
25 hz expert operator’s action data<p>We’re just getting started. We have good amounts of variations in daylight, weather, tasks, and would be adding more hours of data and also converting to lerobot format soon. We’re doing this so people like you and me can try out training models on real world data which is very, very hard to get.<p>So please checkout the dataset here and feel free to download and use however you like. We can’t wait to see what you all build. I’ll be around in the thread and look forward to comments and feedback from the community!<p>Dataset link: <a href="https://huggingface.co/datasets/FlywheelAI/excavator-dataset" rel="nofollow">https://huggingface.co/datasets/FlywheelAI/excavator-dataset</a>