Random USB issues with 4 ODrive v3.6s

I’m using 4 pcs of ODrive v3.6 in order to command a robot. The system voltage is 24V, and the ODrives are of the 24V variant.
A small PC (Jetson Nano, but we also tried others) command the four ODrives over USB. Due to lack of ports, we use a USB hub. The USB cables are 1.8m long, new, from a reputable source.

The problem is that even without doing anything, the ODrives disconnect and reconnect randomly as seen in ODrivetool:

ODrive control utility v0.6.5.post1
Connected to unknown device 2068346E5748 (firmware v0.5.4) as odrv0
Connected to unknown device 208835873056 (firmware v0.5.4) as odrv1
Connected to unknown device 207A34765748 (firmware v0.5.4) as odrv2
Website: https://odriverobotics.com/
Docs: https://docs.odriverobotics.com/
Forums: https://discourse.odriverobotics.com/
Discord: https://discord.gg/k3ZZ3mS
Github: https://github.com/odriverobotics/ODrive/

Please connect your ODrive.
You can also type help() or quit().

In [1]: xz = odrv0

In [2]: xz.axis0.requested_state = AXIS_STATE_CLOSED_LOOP_CONTROL

Oh no odrv1 disappeared
Reconnected to unknown device 208835873056 (firmware v0.5.4) as odrv1
Oh no odrv1 disappeared
Oh no odrv2 disappeared
Reconnected to unknown device 208835873056 (firmware v0.5.4) as odrv1
Oh no odrv1 disappeared
Task exception was never retrieved
future: <Task finished name='Task-3' coro=<_Domain.run_discovery.<locals>.loop() done, defined at /usr/local/lib/python3.11/dist-packages/odrive/pyfibre/fibre/libfibre.py:977> exception=TypeError("Expected value of type RemoteObject or None but got 'EmptyInterface'. An example for a RemoteObject is this expression: odrv0.axis0.controller._input_pos_property")>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/odrive/pyfibre/fibre/libfibre.py", line 980, in loop
    await callback(obj)
  File "/usr/local/lib/python3.11/dist-packages/odrive/pyfibre/fibre/shell.py", line 15, in discovered_device
    mount_result = await mount(device)
                   ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/odrive/shell.py", line 127, in mount
    serial_number_str = await odrive.utils.get_serial_number_str(obj)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/odrive/utils.py", line 52, in get_serial_number_str
    return format(await device._serial_number_property.read(), 'x').upper()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/odrive/pyfibre/fibre/libfibre.py", line 535, in async_call
    tx_buf += arg[2].serialize(self._libfibre, args[i])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/odrive/pyfibre/fibre/libfibre.py", line 247, in serialize
    raise TypeError("Expected value of type RemoteObject or None but got '{}'. An example for a RemoteObject is this expression: odrv0.axis0.controller._input_pos_property".format(type(value).__name__))
TypeError: Expected value of type RemoteObject or None but got 'EmptyInterface'. An example for a RemoteObject is this expression: odrv0.axis0.controller._input_pos_property
In [3]: 

The problem is less pronounced if I only have 1 ODrive connected at a time (I’ve used this successfully to configure them), but for the end application I will need all 4. The problem doesn’t always appear when I enable closed loop control. Sometimes it holds on and I can move the motors, but eventually it would randomly disconnect and reconnect. It seems random, not much related to what I’m doing.
I’m testing on Linux and dmesg shows that the ODrive USB devices disconnect/reconnect, or have read errors.

I’ve read that people use USB isolators to improve USB reliability, but I’m confused how to test it in my scenario. E.g., one isolator per ODrive, or it would be sufficient to use one isolator, and then spawn the 4 through an unpowered USB hub.

BTW, we’ve tried with both powered and unpowered USB hubs, that doesn’t make any difference.

Please, give some advice how to rectify the issue?

I think you’d need one isolator per ODrive. Did you try to connect one ODrive via isolator directly to the Jetson Nano? I did that once and had no USB issues.

I was able to borrow one isolator from a friend.
But he had only one, and more are on order, but will arrive next week.
Anyhow, the results are:

  • One ODrive, through isolator: works fine
  • One ODrive, direct USB: works fine (as noted in the question)
  • Two ODrives, one through isolator, one direct: issues if I command any of the ODrives into closed loop control (the one on the isolator disconnects)
  • [PC]→[Isolator]→[unpowered USB hub]⇉[2×ODrives]: both ODrives disconnect if I command any into closed loop control

Sounds like you have some noise suppression issues. You have ferrite rings on all the motor wires?

You should really switch to CAN.

No ferrite rings whatsoever. Is there a guide on where and how to put them?

I was able to get a somewhat stable config yesterday, with one ODrive connected directly to a USB port (but I used a different USB cable, so the existing may be suspicious), and the other one through the isolator. This works fine driving 4 motors, even at quite high speeds and currents.

It could be a bad USB cable, we’re investigating that. Or since this cable runs along the chasis, near other cables, some of which are high-current, it may be receiving sufficient EMI to cause trouble.

In the long run, we’ll switch to CAN. Can you suggest a good way to interface a PC that is not RPi (so no CAN hats are available)? We were thinking about using a USB-CAN adapter like this Seeed Studio one, but if you’ve had luck with something else, we’ll be happy to follow the beaten path?

You need a USB isolator one for each ODrive, the ground connection in the USB cable can cause several kinds of issues.
You should use USB cables with ferrite cores on them, the one included with the ODrive v3.6 has this so you should be good there.
If you still have issues use the ferrite rings we sell: Ferrite Ring ESD-R-28C-1 — ODrive

Many thanks, @madcowswe!
Given the circumstances, I think it’s better to switch to CAN rather than trying to patch-up the 4xUSB connections with one isolator each, right?

The CAN will be nonisolated, but it should be less of an issue IIRC. As I saw in a similar thread, the suggestion there is to have the GND connection of the CAN link connected to the system ground point, but not on the logic GND pins of each ODrive. I.e. each ODrive only connects to the CAN L, CAN H signals; even the USB-CAN adapter may also use this arrangement (only CAN L, CAN H connected to it, the GND pin remains unconnected). I’m not sure about this last one though.

1 Like

That’s typically how people do it, yeah