copper-rs 1.3: it is a matter of time

This will be our last installment of news about the 1.3 release. Let’s talk about clocks and time for robotics, it is fascinating topic.

Clocks are fundamental for the operation of a robot. This is why time has always belongs to the framework in copper. Every single message comes with a Time of Validity attached as a first class entity representing when any piece of data or derived piece of data is supposed to represent in time.

but time is complicated and even roboticists make a lot of mistakes about it …

Easy, just take the local time…

… people, why all our robots are crashing twice a year at daylight saving changes??

oh ok, UTC is a the one then…

… the clock just displayed 23:59:60 Oo, what the hell, everything crashed!

This time I’ll make it bullet proof GPS time!

… the robot left the building and the time went… backward?!?!? it overflew and controls went full power towards the wall -_-

First thing you need to realize: your robot usually doesn’t care about what time it is on a wall clock.

A robot needs a clock that is (by rough order of importance):

  1. Monotonic: ie. it always go forward in time. Otherwise: this is like injecting a sign error when it goes backward… forward is backward, left is right, up is down ….

  2. Continuous: no sudden stop and jump forward neither. Otherwise: every becomes frozen for a time then and suddenly jerk forward.

  3. With a Local Stable Rate: even with 1 and 2, imagine one second is actually 2s and the next second is half a second, your robot will have a bad day computing speeds.

  4. Fine resolution: with 1 and 3, cool but what if the clock only updates every seconds. A little bit like 2, you observe a jump in time.

  5. Bounded uncertainty: imagine you need to compute a before vs after event difference, your robot needs to know if the 2 timestamps are too close to not overlap otherwise you might end up violating 1 because even if the clock itself was monotonic, the actual physical points in time they were taken might have been reversed just by the measurement error.

  6. Low read overhead: you can respect 1,2,3,4,5 but if it takes 2s or worse a random time to actually read the clock, this might backfire quickly!

(referenced as #1 to #6 below)

The Copper RobotClock before 1.3

So what did we do for our RobotClock in copper? We just took “your robot usually doesn’t care about what time it is on a wall clock” as a first principle and implemented the RobotClock strictly based on CPU Time Stamp Counters (TSC).

A modern CPU is fast… hell fast. Think about a 5GHz CPU, it needs a clock as a signal clock or think about it as an orchestra conductor that mark the beat for everything in the CPU to do “the next thing” 5 BILLION times a second. It is usually derived from what the TSC and in the order of a GHz for modern architectures, in the ns precision range!

As not Intel, AMD and Arm have stable TSC that guarantees 1, 2, 3, 4 and 5 … guess what did we used for our RobotClock!

For the curious people, we use directly some assembly to read it to be sure as less as possible is in between your ctx.now() and the result:

#[inline(always)]
pub fn read_raw_counter() -> u64 {
    let mut counter: u64;
    // SAFETY: Reading the virtual counter register is a side-effect-free CPU instruction.
    unsafe {
        core::arch::asm!("mrs {}, cntvct_el0", out(reg) counter);
    }
    counter
}

but we have all those implementations for all the platforms we currently support:

core/cu29_clock/src/raw_counter/wasm_instant.rs
core/cu29_clock/src/raw_counter/riscv64_rdcycle.rs
core/cu29_clock/src/raw_counter/x86_64_rdtsc.rs
core/cu29_clock/src/raw_counter/armv7r_pmu.rs
core/cu29_clock/src/raw_counter/aarch64_cntvct.rs
core/cu29_clock/src/raw_counter/std_system_time.rs  # for anything we don't support yet :)
core/cu29_clock/src/raw_counter/cortex_m_dwt.rs

Note we actually calibrate the robotclock on this raw counter very quickly this clock with an actual system clock second so it is close to a second.

Cool but what if you have sensors? other robots? a swarm robots?
He comes the clock-sync feature of copper-rs 1.3!

So yes, we simplified the first version to be sure we did not get it wrong and this is was not a major issue because determinism & replay in copper is independent from the clocks … you read that right, this is why you will never make it replaying ROS Bags ever, it is based on time and not causality!

But we improved the situation with a clock-sync feature that allows you to peg the RobotClock to an external source preserving #1, #2, #3, #4, #5 and #6!
adding 3 new properties for a synchronized clock:

7. Holdover protection: when you lose your reference clock it needs to keep on going
8. Common reference clock: you need a common clock across several systems
9. Explicit Identity: you cannot mix and match clocks and you know where they are coming from and their state

(referenced as #7 to #9 below)



As usual in copper we push forward an API that you can implement, clock-sync does that but doesn’t prescribe the actual source of the synchronization.

PTP synchronization reference

Out of the both we offer a RobotClock <-> PTP synchronization. PTP is a very good ISO syncronization standard, implemented in a lot of sensors, first class in Linux, a lot of ethernet controllers can timestamps from a time source to emit PTP sync packets etc!

If you want to see how to implement that, check out our full clock sync example on Linux.

So the local RobotClock with the local PTP sync is made through a resource you declare as a PTP clock respecting all the 9 properties we mentioned earlier.

What about the PTP to PTP sync? This is up to you! You can implement it with ethernet and LinuxPTP, sharing a PPS signal, peg it to a GNSS time to get an atomic clock in your robot, it should be relatively straightforward!

See the clock hierarchy on the robot

The new ConsoleMon's CLK panel now shows the selected reference, upstream source, epoch, synchronization state, offset, and uncertainty. A GNSS source carried through PTP is identifiable as such: the origin of the time and the transport carrying it are separate information.

The clock is a first class citizen, it has an identifiable origin and visible synchronization quality.

For the kicks … what about my time in the ROS2 world? 🤦

Of course copper-rs is building on top of the shoulder of a GIANT ROS 2 clock and time design… wait what?

“The ROSTime will report the same as SystemTime when a ROS Time Source is not active” → ie #1, #2, #3, #4, #5, #6, #9 is out of the box, out of the window!

“In nodes which require the use of SteadyTime or SystemTime for interacting with hardware or other peripherals it is expected that they do a best effort to isolate any SystemTime or SteadyTime information inside their implementation and translate external interfaces to use the ROS time abstraction when communicating over the ROS network.” → #8 is actually always out of the window, user, figure it out!!

ROSTIme is the one for sure right? “The time abstraction can be published by one source on the /clock topic. The topic will contain the most up to date time for the ROS system. If a publisher exists for the topic, it will override the system time when using the ROS time abstraction. If /clock is being published, calls to the ROS time abstraction will return the latest time received from the /clock topic. “ ie. #2, #3, #4, #5, #7 and #9 are dead already.

[...]

It is possible to use an external time source such as GPS as a ROSTime source, but it is recommended to integrate a time source like that using standard NTP integrations with the system clock since that is already an established mechanism and will not need to deal with more complicated changes such as time jumps.” ie. check the no no no no no wait wait wait meme attached.

Back to sanity …

The clock synchronization guide explains it all to use it with your copper instance.

We can’t wait to see what you are building with copper-rs! Check out and share on our newly minted subreddit r/copper_rs !

Next
Next

copper-rs 1.3: self-describing logs and log rollover on robot and on twins