Humanoid Robot Benchmarks: What Buyers Should Ask

That is why two recent benchmark efforts matter. NIST's Humanoid Robot Baseline Performance Benchmark is aimed at minimum expected physical capabilities for commercially available humanoids. Fraunhofer IPA's humanoid benchmark goes further into application criteria such as functional safety, cybersecurity, cleanroom suitability, and energy efficiency. Both are written for a market that has outgrown edited highlight reels.

For home buyers, the useful question is not "which benchmark will crown the winner?" It is: which benchmark questions should you demand before letting a bipedal robot walk near furniture, pets, children, stairs, glass doors, and expensive stuff?

The short version: do not buy a home humanoid because it can do one chore once. Buy only when the maker can show repeatable mobility, repeatable manipulation, measured contact forces, real battery behavior, and a clear failure mode for when the robot gets confused.

Why Benchmarks Suddenly Matter

NIST says the last period when humanoid performance was rigorously measured across robots was the DARPA Robotics Challenge era in 2013 and 2014. That matters because the market in 2026 is no longer just university labs. Buyers can already find real products, preorders, rentals, and enterprise humanoids with home-style promises attached.

In the ui44 database, Unitree G1 is the clearest example of a humanoid you can actually point to as available. It starts at $13,500, stands 132 cm, weighs 35 kg, and lists about 2 hours of battery life. 1X NEO is a $20,000 preorder at 167 cm, 30 kg, and roughly 4 hours of battery. AGIBOT Expedition A3 is listed around $45,000, stands 173 cm, weighs 55 kg, and claims up to 10 hours with a dual hot-swappable battery system.

Those specs are useful. They are not enough. A spec sheet can tell you whether a robot is small enough for a hallway or heavy enough to be a concern. It cannot tell you whether the robot can recover from a bad grasp, stop before a collision becomes painful, or repeat a task on a Tuesday after a software update.

What NIST Is Trying To Standardize

NIST's proposal is important because it starts with a modest but powerful idea: establish a low-footprint baseline for locomotion and manipulation. The point is not to prove that a humanoid can run a perfect household. It is to make basic physical capability measurable across robots.

The proposed task set exercises four areas that translate directly to homes:

The word "baseline" is doing a lot of work here. A home robot buyer should not expect NIST's first baseline to answer every consumer question. It will not tell you whether the robot is kind to a nervous dog or whether it can put away your exact mugs. But a baseline can make weak claims easier to spot. If a humanoid cannot pass a public low-footprint mobility and manipulation test, it has no business being marketed as a general-purpose home helper.

The most useful buyer demand is simple: show the score, not just the clip. A clip shows what happened once. A score should show how often it happened, under what setup, and what counted as a failure.

What Fraunhofer Adds: Safety, Security, And Runtime

Fraunhofer IPA's benchmark is more application-focused. It groups tests into technologies and basic abilities, complex abilities, cleanroom suitability, functional safety, cybersecurity, and energy efficiency. For homes, three of those categories deserve special attention: contact safety, remote-control security, and runtime under motion.

The Fraunhofer example is concrete because the institute applied the benchmark to a Unitree G1 EDU-4 delivered in May 2025 with Dex3-1 three-finger hands and firmware version 1.04. The results were not a generic thumbs-up or thumbs-down. They exposed the kind of detail buyers usually never see.

Fraunhofer reported collision forces over 500 N in some cases, a critical Bluetooth remote-control vulnerability in the software version tested, and maximum battery times of 2 hours 49 minutes while standing and 1 hour 49 minutes in a stand-walk scenario. Fraunhofer says the Bluetooth issue has since been fixed, which is exactly the point: benchmarking is useful because it finds problems early enough for vendors to fix them.

For a home buyer, those numbers change the conversation. A robot that weighs 35 kg and can generate high collision forces is not just a gadget with legs. It is moving machinery in a shared space. A robot with a remote-control weakness is not just a privacy issue. It is a physical safety issue. A robot that lasts under two hours while standing and walking is not ready to be treated like an all-day household worker unless it can dock, pause, or hand off safely.

The Evidence Ladder Buyers Should Use

Not all proof is equal. A five-second clip, a shipping page, an internal test, a public benchmark, and an independent lab result should not carry the same weight.

Use this ladder when comparing humanoids:

1. Edited demo: Good for curiosity, weak for purchase decisions.
2. Shipping spec: Better, because price, size, weight, payload, and battery can be compared.
3. Repeated task evidence: Stronger, especially if failures are counted.
4. Standardized benchmark: Stronger again, because the method can be repeated.
5. Independent benchmark: Best, because neutral testing reduces the chance that the environment was designed around one robot's strengths.

This is why Figure 02 and UBTECH Walker S2 should be evaluated differently from a home preorder. Figure 02 has industrial proof and a large 20 kg payload, but it is not publicly priced and is now marked discontinued in the ui44 database. Walker S2 has a 15 kg payload and autonomous battery swapping aimed at continuous enterprise operation, but no public consumer price. Those facts are meaningful, yet they do not prove a robot can safely work in a home.

How This Changes The Robot Comparison

Benchmarks force a buyer to ask different questions. "Which humanoid is cheapest?" becomes "which humanoid can prove safe contact, repeatable manipulation, and predictable failures at its price?"

The table is deliberately skeptical. A robot can be impressive and still be a bad home purchase. A low price can make experimentation possible, but it does not remove the need for safety data. A long battery claim can sound useful, but it does not prove that the robot can work for hours while manipulating real objects.

Where Competitions Fit In

Benchmarks are not only coming from standards labs. AGIBOT's World Challenge 2026, announced alongside ICRA 2026, drew 526 teams from 27 countries across Reasoning-to-Action and World Model tracks. The useful signal is not the trophy. It is the shift from simulation-only scores toward real-robot finals, standardized metrics, task planning, physical execution, and failure cases such as object drops and grasping problems.

That kind of competition is still not the same as a home safety certification. But it can reveal whether the software stack is improving on the messy middle: language understanding, spatial reasoning, disturbance adaptation, long-horizon task reliability, and mobile manipulation under physical constraints.

For buyers, competitions should be treated as supporting evidence. If a company says its model did well in a real-robot benchmark, ask what robot body was used, what tasks were run, how many attempts were allowed, and whether the task looked anything like a home chore.

What should home buyers ask before trusting a humanoid robot?

Before putting a deposit on a humanoid robot, ask for evidence in five areas.

Mobility: Can it walk on ordinary flooring, turn in a hallway, handle thresholds, and stop safely near stairs? A home robot that only walks on open showroom floors has not proved enough.

Manipulation: Can it pick, carry, place, and release common objects with measured success rates? Hands and arms matter more than dance moves. A useful home robot needs to handle light, awkward, and fragile things without relying on perfect placement.

Contact safety: What forces were measured during collisions? What happens if a person bumps the robot, a pet crosses its path, or a child grabs an arm? Fraunhofer's G1 test is a reminder that contact force needs numbers, not adjectives.

Security: How are remote control, Bluetooth, Wi-Fi, updates, logs, and cloud access protected? A vulnerability on a screen is bad. A vulnerability on a moving robot is worse.

Runtime and recovery: How long does the robot run while moving and manipulating, not just standing? Can it dock itself, swap batteries, pause safely, and recover after a failed grasp?

If a seller cannot answer these questions, the honest conclusion is not "the robot is fake." It is "the robot has not yet earned home trust."

What To Watch Next

The most useful future benchmark will combine NIST-style baseline tasks with Fraunhofer-style safety, security, and energy tests. For homes, it should also add plain-language reporting: number of attempts, success rate, maximum contact force, time to complete, human interventions, battery used, and the exact environment.

That would let buyers compare a $13,500 Unitree G1, a $20,000 1X NEO preorder, a $45,000 AGIBOT platform, and enterprise humanoids on something more meaningful than edited video. It would also help separate two very different claims: "this robot can do a task once" and "this robot can be trusted to repeat that task around people."

Until then, use benchmarks as a filter. The best home humanoid is not the one with the most cinematic demo. It is the one whose maker can say, in public, how it was tested, how often it failed, how hard it hit, how long it ran, and what it did when the task went wrong.

That is the point of benchmark culture. It turns "look what our robot can do" into "here is what our robot can prove."