Unresponsive USB after about 20 secs


#1

Hello there,

I’m struggling to use the USB interface in a reliable way.

Basically sending back and forth positionning commands results in the board going unresponsive. I experienced it with the first odrive prototypes in april. Then again with 3.2 very recently using the latest code available.

I have documented my attempts there https://github.com/madcowswe/ODriveFirmware/issues/17 [EDIT: was moved to below instead]

Has anyone experienced anything similar ?

Thanks


#2

Hi @alexisdal,

Thanks for reporting the issue! I saw you posted the details of the issue on that link on github, but that issue is actually about cleaning up and enhancing the protocol: sorry about the confusion. I copied your report to here instead:

i just reproduced the issue using

  • odrive board v3.2
  • turniny aerodrive 5065 236
  • a simple lasercut hardward jig to hold the motor and the encoder.
  • a rohde&shwarz lab power supply (settings on 24v / max 4a)
  • a R47 resistor connected on AUX (since i am using a power supply instead of a battery, my understanding was that i should use a “big” resistor connected on AUX)
  • lastest master code (06047cb) updated by Oskar 6 days ago. no modification at all.
  • the same code snippet as above (basically sending “p 0 9999 0 0\n” and “p 0 -9999 0 0\n” back and forth every 3 seconds)
  • After about 20~30 seconds, the motor stops moving and usb interface no longer responds. the only way to work with the board is tostop sending request, turn off and on power supply, wait for the init sequence to be over.

I don’t care much about the API interface as long as it is simple and stable. what madcowswe suggests above is very fine for me. I have not tried the step/dir hardware interface yet.

Basically the symptom is the USB interface is not usable.

Should I use another motor?
Should I use another code branch?
Should I use the step/dir interface instead of USB?
Does anyone reproduce the issue?


#3

What OS and driver are you using? If you are using Windows, did you use Zadig to change the driver to libusb, as is explained here?
How are you sending commands, via a python script and usbbulk.py, or through some other means?

On Windows the CDC driver is very unstable (at least with how ODrive has it set up), and it would be convenient to get it to be stable so you don’t have to switch drivers, and you can use existing terminal programs to send commands; but for now we are using libusb instead as it seems to work much better.


#4

i’m using a standard ubuntu desktop, both to compile the firmware and to send command on USB. it’s probably a 16.04, i can check but it’s likely to be off topic. I have not tried piloting the odrive from windows since it’s not likely to be use case in the future (but i can if it helps troubleshoot the issue)

To send command on USB, i simply send text commands to /dev/tty.ACM0 from the system terminal.

echo -en “p 0 9999 0 0\n” > /dev/ttyACM0

then i merely used a bash loop to repeatedly send such commands every 3 seconds or so.

I can try to reproduce the behavior using a python script instead of shell commands.

=> would that help ?

thanks


#5

Ah okay cool. Yeah please do try from the python. It uses pyusb drivers instead of the CDC drivers. You can have a look at test_bulk.py to see how it sends the commands. I’m keen to see if this is more stable, thanks for testing!


#6

i finally managed to get a moment to try this out.

i modified the code of test_bulk so that instead of prompting user for a command i would provide the input directly from script

steps = 9999
while running:
  time.sleep(0.1)
  try:
      #command = input("Enter ODrive command:\n")
      command = "p 0 "+str(steps)+" 0 0"
      dev.send(command)
      steps = -1 * steps
      time.sleep(3.0)

the good news is that my turnigy 5065 236 has been spinning back and forth for about 5minutes now. however, it’s kinda weird because the system draw very little current (at 24V: 0.03A when the motor does not spin vs 0.17A when it does). I suspect my colleague has reflashed the board with default parameters that are designed for small motors (to prevent burning such motors of a new user if i remember right)

I will let the system turning back and forth for the entire night

oskar, would you by any chance know of a given set of parameter for my motors that are likely to work better (and yes I will go through the motor params procedure in the README section later).

Thanks


#7

nope… it stopped spinning after about 10min.
And the noise of the motor is weird, it looks like it does not receive enough power (same noise my son’s remote controlled toy car makes when it’s very low on batteries). but i’m operating it with a power supply connected to main power (Rohde & Schwarz DC POWER SUPPLY NGSM 32/10)

besides, once the motor no longer moves, if i try to move the motor by hand as the script continues to run, the motors will try to spin but will not spin back. so it looks like the USB communication does work but the board does not provide enough power for the motor to spin.

i tried with 18V but observed same behavior.

oskar, would you have a quick an easy motor params for my turnigy aerodrive 5065 236 by any chance ?


#8

We did recently find a bug that causes intermittent deadlock on the USB communication thread. This would make the ODrive unresponsive on USB until you reboot it.
I just released a patch update fw v2.1, which is now pushed to master. See if you can flash this, and if that fixes your communication issues?

However the issue you describe about the weird noise in the motor I’m not sure is related. Could you possibly record a video showing what’s going on? I am also happy to do live support over Skype if you prefer. I’m also in the ODrive Discord chat:


#9

just updated to commit a7b11f8c6db30af55302b8b0343bd9b601579208
(see the other thread to fix the compilation error, which is gone)

unfortunately i observe the same behavior. after about maybe 20 to 40 commands sent through USB, the motor no longer moves. To be more precise, i modified the script to show USB errors (see my modification to test_bulk.py below)
the
’ODriveBulkDevice’ object has no attribute ‘recieve’

seem to happen all the time, regardless of the motor actually moving or not.

So it really looks like it is not really related to the USB communication channel, but more something about the board itself. Besides, it the last working command execution (or maybe the 2 latest commands), the motor makes a slightly different sound , as if running out of battery (even if i connect it to a power supply) or as if it would spin slower. it’s hard to say but the sound is clearly different.

maybe I should adapt default motors parameters (i did not touch that part). could it be related ?

I’ll shoot and post a video of the behavior soon.

root@MTBD00694:~/ODriveFirmware/Firmware# cat ../tools/test_bulk2.py 
#! /usr/bin/env python3

import argparse

def parse_args():
  parser = argparse.ArgumentParser(description='Talk to a ODrive board over USB bulk channel.')
  return parser.parse_args()

if __name__ == '__main__':
  # parse args before other imports
  args = parse_args()

import sys
import time
import threading
from odrive import usbbulk

running = True
ready   = False

def main(args):
  global running
  print("ODrive USB Bulk Communications")
  print("---------------------------------------------------------------------")
  print("USAGE:")
  print("\tPOSITION_CONTROL:\n\t\tp MOTOR_NUMBER POSITION VELOCITY CURRENT")
  print("\tVELOCITY_CONTROL:\n\t\tv MOTOR_NUMBER VELOCITY CURRENT")
  print("\tCURRENT_CONTROL:\n\t\tc MOTOR_NUMBER CURRENT")
  print("---------------------------------------------------------------------")
  # query device
  dev = usbbulk.poll_odrive_bulk_device(printer=print)
  print (dev.info())
  print (dev.init())
  # thread
  thread = threading.Thread(target=recieve_thread, args=[dev])
  thread.start()
  steps = 9999
  sent_cmds = 0
  while running:
    time.sleep(0.1)
    try:
      #command = input("Enter ODrive command:\n")
      command = "p 0 "+str(steps)+" 0 0"
      dev.send(command)
      steps = -1 * steps
      sent_cmds += 1
      print(str(sent_cmds))
      time.sleep(3.0)
      
    except:
      print("there was an error in the main loop...")
      running = False

def recieve_thread(dev):
  global ready
  while running:
    time.sleep(0.1)
    try:
      message = dev.recieve(dev.recieve_max())
      message_ascii = bytes(message).decode('ascii')
      print(message_ascii, end='')
      if "ODrive Firmware" in message_ascii:
        ready = True
    except Exception as e:
      print("there was an error in the receive_thread() function: "+str(e))
      pass

if __name__ == "__main__":
   main(args)
root@MTBD00694:~/ODriveFirmware/Firmware#

#10

Here’s a video with the behavior.

look in particular at about 00:12 and 2:30
(unfortunately someone called me on my mobile while I was shooting the video with it so the recording stopped before the behavior would repeat a 3rd time)

it’s so frustrating :cry:

How can I move forward? Should I touch the default motor parameters? (reminder I have mounted a turnigy aerodrive 5065 236)


#11

Hi @alexisdal,
Sorry that it’s so frustrating for you, I assure you we will make sure you get it working.
Thanks for the video, this really helps. I agree with you that it seems it’s not a communication problem, since nothing changes. I can also see on the power supply current draw that the motor is trying to go, but “runs out of power” like you say.

This makes it really look like the encoder is slipping: either mechanically, or electrical noise is making false counts.
Can you please check very carefully that the coupling between the motor and encoder is good? Otherwise, we need to check with an oscilloscope the signal integrity of the encoder pulses.

Thanks for your patience!
Cheers,
Oskar


#12

no problem i’m determined to go further.

I belive the mechanical coupling should be ok because if i remember well

  • on one side the encoder’s shaft is tapped (it is not entirely round, there is a flat area, the profile is like a D). so with two screws, I doubt it can easily slip
  • on the motor side, it’s shaft is round (profile is a circle and not a D). I beleive we made a small hole with a drill so that the screw would physically lock the shaft and could not slip. But i’m unsure. will check asap

However, it was 3D printed, I had to drill it to make sure the size of the shaft would be ok. Beside, depending on how you turn the screws the shafts may be not perfectly aligned (i did try to make a nice alignment though. it’s probably not perfect)

i will try to change the encoder too.
the one I have comes from aliexpress
https://www.aliexpress.com/item/Free-Shipping-Encoder-600-P-R-5V-24V-Incremental-Rotary-AB-2-Phase-6mm-Shaft-Coupling/32641634967.html
I ordered two units just in case.
and two “more serious” encoders from CUI
Digital Encoder Radial CUI AMT10x
490-AMT102-V
http://www.mouser.fr/ProductDetail/CUI/AMT102-V/?qs=sGAEpiMZZMvy8cVzszrmR8e0mUKZ7grDKaNMqTNrY3iTjPe3k%2bpUaQ%3D%3D

This one seems to be much better (much more compact). but i just didn’t know how to mount them and didn’t know for cabling either.

i have never had to use an oscilloscope to check the functionality of the encoder.
What pins should i connect to the oscilloscope then?

by the way, the resistor is
https://www.digikey.com/product-detail/en/te-connectivity-passive-product/HSA50R47J/A102181-ND/2056131
(don’t know if that can have an influence as well?)
I may have another unit (in general i order by 2 to be able to swap them in case of failure)
I must have another physically looking resistor with other values. must check first what we have in drawers.

hope that helps


#13

Hi,
Thanks for all that detailed information.
You can order a metal coupling for cheap from Aliexpress: Link. Of course the shipping takes quite a while, but they do a better job than a 3D printed one.

Let’s try with a verified or changed encoder coupling, or even swapping encoders, first, before we go into checking with the Oscilloscope.
The AMT10x encoders use normal 2.54 male pins, you can use standard female jumper cables to interface.


#14

i think i nailed the problem.
the mechanical connection between the motor’s shaft and the coupling was slipping after a while. Unlike what i remembered the shaft was not tapped/drilled, so the screw was slipping on the shaft. so basically the odrive would most likely send micro-impulsions to the motor, but would observe no movement => it probably explains the behavior.

I tapped the motor’s shaft with a power tool and now it no longer slips even if i brutally force it.

About 250 commands have been sent now (and counting). i’ll let it roll for en entire night, maybe the entire weekend (do you think there is a risk of fire? the cables are cold/normal).

Do you think you could change the software so that the odrive returns an error code in case something’s wrong? or is it bad idea? (my guess is that most users wouldn’t read error code no?)


#15

Ok great! I can’t guarantee anything about fire risk. Personally I wouldn’t be too scared to leave it overnight; especially if everything is left on a non-flammable surface, like stone floor or concrete.

Cheers,
Oskar


#16

it’s on a desk. it continued the test all night long and today as well.
over 26 000 commands.
no particular problem
no heat anywhere (not on the cables, not on the resistance, not on the motor (i dont dare touching the board coz i’m scared of making shorts)

i think i’ll keep it turning all weekend.

i’m building trust with ordive now :smiley:

next week i have a 2 hours time windows on monday afternoon. i’ll use it to follow the procedure and compile a low_leve.c with values pushing the limits (instead of defaults values where only 0.200A are consumed). then will let it spin back and forth like this for an entire week
Then i’ll build a 2 motors jig to simulate some GCODE moving both motors to random XY locations…

  • add some physical load/weight on the motors shaft with pulleys and continue the testing…

#17

330 000 commands and still going without any apparent issue.
the motor has been spinning back and forth 24/7 for about 12 days now.

i hope i’ll move on to something more serious soon


#18

Thanks for taking the time to do this endurance test. Just for reference, can you tell us what git commit you were running? The command to find out is git describe --always. Thanks.


#19
root@MTBD00694:~/ODriveFirmware# git describe --always
a7b11f8

393 000+ commands and still spinning. i did not have the time to change the testing environment (compiling fw for more suited motor parameters + adding weight/pulleys on the shaft. So i let it spin back and forth.

i find it very stable.

I can observe that while spinning in one direction (clockwise) the stop is a quick cut. However, in the other direction (counter clockwise), as it’s about to stop the movement, it continues the movement for i’d say maybe 10 degrees. But it’s been like this ever since i started playing with odrive and that motor + it’s systematic. I’m guessing that once I use more suited params (basically with more torque), the behavior should vanish?

=> Do you want a video of the behavior?


#20

Thank you.
Yeah it makes sense that for un-tuned parameters you will get some overshoot, and yes it’s normal no have slight asymmetry, there are many sources of small disturbances. My guess is that the motor cogging torque is the contributor here.
No need for a video, just good to know that the communication seems to be rock solid.