I'm trying to understand where the biggest supply gaps still exist in real-world data for robotics and embodied AI. I'm not referring to synthetic or simulation data, only data collected from the physical world. Some examples I'm thinking about: Dexterous manipulation Tactile/contact sensing Bimanual tasks Warehouse/logistics Industrial assembly Mobile manipulation Long-horizon household tasks Human demonstrations vs. robot-generated data For those working in robotics or VLA/world model research: What types of real-world data do you wish existed in much larger quantities? Are there specific verticals (manufacturing, healthcare, retail, agriculture, etc.) where data is especially scarce? Are there modalities (RGB, depth, tactile, force, audio, IMU, eye gaze, etc.) that are consistently missing? If someone were starting a company focused on collecting real-world robotics data, where would you say the biggest unmet need is today? I'd love to hear perspectives from anyone training robot foundation models, collecting datasets, or deploying robots in production. submitted by /u/Tricky-Promotion6784 [link] [Kommentare]
This new paper develops an information-processing theory of consciousness and uses it to identify how consciousness can be instantiated in AI, paving the way for genuine AGI and beyond (the paper demonstrates that conscious functioning is the missing ingredient that enables a toddler to navigate an obstacle-strewn room or an 18-year-old to learn to drive with massively less training than is required by a robot or autonomous vehicle): Abstract: An acceptable information-processing theory of consciousness should be able to identify the adaptive advantages that drove the emergence of consciousness during the evolution of life. It should also predict the specific dynamical architecture of information processing that would need to be instantiated in AI to produce consciousness and the superior adaptation it enables. Whether such an instantiation produces AI that is actually conscious and also more adaptable would provide the ultimate test of the theory. A prime candidate for such a theory is the Subject-Object Emergence Theory of consciousness. It argues that consciousness first evolved because it enabled organisms to achieve adaptive body-environment coordination without extensive trial-and-error learning. It postulates that the subject in an appropriate Subject-Object subsystem would be able to use depictive (iconic) visual representations of the relative positions of its body and the environment to guide motor actions that will produce adaptive body-environment coordination. The depictive representations will 'light up' for such a subject, producing subjective experience that is used to deliver adaptive benefits. Hand-eye coordination is a familiar example in humans—novel and intricate coordination tasks can be undertaken without additional reinforcement learning, provided focused conscious attention is employed to provide us (the subject) with relevant depictive images. The paper identifies how such a conscious Subject-Object subsystem could be instantiated in AI systems, enabling hand-eye and other body-environment coordination without the extensive reinforcement learning or complex computational programming needed at present. Drawing further on the Subject-Object theory of consciousness, the paper also identifies how these simple conscious subsystems evolved further in organisms to establish the conscious modelling that enables conscious planning, imagining, abduction and other higher cognitive functions. It demonstrates that current approaches to incorporating world modelling in AI will fail to achieve key elements of the general intelligence found in humans that require consciousness. The full paper can be accessed freely at: https://ssrn.com/abstract=6911039 submitted by /u/BigPicturexyz [link] [Kommentare]
I’m trying to build a model evaluation platform and wanted to see which ones I should evaluate first. Any resources on this would be helpful. submitted by /u/No-Bad-583 [link] [Kommentare]
Trying to separate demo-reel hype from actual capability. Every few weeks a new humanoid video goes viral, but polished demos and deployed systems are very different things. submitted by /u/futurewilltell [link] [Kommentare]
Robbyant released LingBot-VLA 2.0, a single model driving 20 embodiments from single-arm Franka and dual-arm UR7e up to full humanoids like Unitree G1 and Fourier GR-2. The action space also covers head, waist, mobile base, and dexterous hands, not only dual-arm manipulation. Training data is roughly 50,000 hours of real-robot trajectories across those 20 configs plus 10,000 hours of egocentric human video, filtered and reconstructed. The ablations on 4 GM-100 real-robot tasks show a clean result: relative joint actions over absolute lifted average success from 33.7% to 55.0%, with relative joint positions cutting the action standard deviation to roughly a third of absolute (about 0.28 vs 0.80). On its own GM-100 generalist eval, Robbyant self-reports higher progress and success than pi-0.5 and GR00T N1.7. Absolute success remains low: 34.4% on Agilex and 15.6% on Galaxea, with several tasks at 0%. The paper itself notes the model often makes partial progress then fails at the final precise placement or release. OOD performance degrades sharply. submitted by /u/deepmoss47 [link] [Kommentare]
I have experience in deploying traditional robotics software, but I feel I am not up to date with what is happening in the modern robotics work like using vlm, vla, etc. People talk about it and show proof of concepts, but never seen a real deployment of these used cases especially in the case of the manufacturing industry. Has anyone deployed a robotic product with a modern robotics AI stack? Would love to gain the insights on this. submitted by /u/Live-Inspector-2641 [link] [Kommentare]
I unfortunately cannot get my hands on a rgbd/ stereo camera. So, I have to find a way with monocular camera. Is there any way? Currently, I have a fully functional ros2 nav2 robot with 2d lidar. How about using depth_anything or such? Can anyone provide me insights? submitted by /u/Candid-Scheme1835 [link] [Kommentare]
So I want to perform a material characterization study on a material where I need to put it under pressure. I’m in high school and don’t have a mentor or time to ask for access to university labs so I want to make something that can help me get data for cheap. I’m trying to make a linear actuator design and physically build all the parts myself (except for the motor and leadscrew system obviously) but I don’t extensively know how these types of things work. If I was to build something like this (pictures) would there be any significant issues? The cylinder (of which I don’t know what material to make out of) protruding out from the side would be directly connected to the sliding block part of my linear actuator so it pushes that down onto my material. I’m going to be pushing with 50lbs ish max so I’m making the majority of this out of wood. Any tips on making sure it doesn’t get worn out by some slight imperfection over the thousands of trials I’m going to need it for? And also any tips to make it work if something is seriously wrong 😭 And lastly any other tips about doing research studies like this without lab access or a significant mentor would be greatly appreciated. submitted by /u/bount_ [link] [Kommentare]
We have spent the past few weeks carefully annotating videos and experimenting with VLMs for subtask annotation. This type of annotation is incredibly important for long-horizon tasks, since robots need a more granular learning signal than high-level instructions like “clean your room.” We ran 50+ experiments, created a new diverse benchmark for this type of annotation, and built a pipeline that is 19x cheaper than humans. It works well as a first pass for labeling, speeding up human annotation and making it substantially cheaper. Blogpost about it is here: https://macrodata.co/blog/annotating-robot-video-subtasks submitted by /u/Other_Housing8453 [link] [Kommentare]
Hi everyone, For the past couple of weeks I have been working on a simulator project considering the shortcomings of MuJoCo. There are things that people like and also don't like about MuJoCo, like the CPU dependency on MuJoCo which makes the simulation not parallelizable beyond a certain limit (depending on the hardware). I know there exists MJX which is GPU accelerated, however, it is not really made for vision based RL pipelines and training. There is also NVIDIA Isaac ecosystem, but that requires a powerful GPU, thus making it limited in terms of accessibility, let alone it requires license. This is why I worked out this new simulator (still working on it, so there will be significant bugs which require fixing). I call it **MuJoFil** \- MuJoCo + Google's Filament Render Engine. Basically I used Nvidia's Newton Physics Engine (which itself is based on MuJoCo's physics engine but is GPU native), clubbed it with Google's Filament render engine (both of these are open-source), modified Filament significantly to support working natively on GPU to render multiple simulations in parallel, and worked on optimizing it for performance. So what is MuJoFil? It is supposed to be an open-source high visual fidelity simulator optimised for a highly parallelized RL training pipeline so that users can use it to train Vision based Policies. Besides, it offers PBR textures support and also a simple to use plug and play functionality, where you can use any environments available online and support formats such as GLB, OpenUSD, etc. for setting environments for your robots. Basically, now you aren't just limited to environments native to MuJoCo, but rather you can use any environments available online from sketchfab, polyhaven, etc. and use it as a practical robot simulation environment. Check it out for yourself in the video. I would really appreciate it if you guys could tell how you feel about it and suggest ideas for what all things I can incorporate into it as this is going to be a fully open-source and free to use simulator that I have been working on for weeks. PS: While I have a couple of published research papers at top RL and AI/ML venues in the field of RL, I still consider myself a learner in this field who is continuously trying, learning, and building stuff, so there will be things in this hugely ambitious project which I might have missed to work on, and that is where I want help from you people who understand this field well. Sorry for this lengthy post and thanks if you read it till here🙇🙇🙏, I would really appreciate if you could share your thoughts on it. Also, I will make its code repo public on GitHub, but till then you can definitely check it out on PyPI. There are 2 separate packages, one can be installed using: "pip install mujofil" This is the CPU based variant, whereas there is a CUDA supporting GPU native variant about which I mentioned above, you can currently install it using: "pip install mujofil-warp" I am planning on changing its name to mujofil-cuda instead of mujofil-warp as that apparently sounds more intuitive to my direct peers but you can suggest this name as well. Thank you for the support❤️. submitted by /u/MT1699 [link] [Kommentare]