ERROR_CURRENT_UNSTABLE with hall sensors

At this point in time I am at a loss of what is going wrong with ODrive and my motor. No matter what I have done I am getting ERROR_CURRENT_UNSTABLE when trying to operate in closed loop velocity control with my motor. The weird thing is that with 0.4.12 firmware I can run in sensorless mode and as long as my acceleration is not too high I can get up to well over 35000 RPM. With closed loop control I am lucky to get above 10k. With 0.5.1 firmware I can not get above 14000 RPM and simply cannot run closed loop. At first @madcowswe and @Wetmelon suggested that the motor’s back EMF was to blame. This is what that waveform looks like on an Oscilloscope though and running at around 19000 RPM though:

This looks very sinusoidal to me. The next thing to rule out was the Hall Sensors themselves. Perhaps they were giving a weird signal. This is what those look like on a scope though:

Hall A and B:

Hall A and Z:

Both of those seem to be working just fine. The last thing suggested was that the motor’s inductance was possibly too low so I soldered three of these in line with the phases:

https://www.digikey.com/product-detail/en/bourns-inc/PQ2617BHA-220K/PQ2617BHA-220K-ND/

That did make it so that on the 0.4.12 firmware I could run at a bit higher acceleration without getting ERROR_CURRENT_UNSTABLE, but it did not help with closed loop control. At this point I am out of ideas and well outside my level of knowledge on motors. @Wetmelon suggested that there could be an error in the hall code affecting this, but this is not something I can debug due to my lack of knowledge on brushless motor theory.

Here is my 0.5.1 config:

In [1]: odrv0.axis0
Out[1]:
error = 0x0100 (int)
step_dir_active = False (bool)
current_state = 1 (int)
requested_state = 0 (int)
loop_counter = 484246 (int)
lockin_state = 0 (int)
is_homed = False (bool)
config:
  startup_motor_calibration = False (bool)
  startup_encoder_index_search = False (bool)
  startup_encoder_offset_calibration = False (bool)
  startup_closed_loop_control = False (bool)
  startup_sensorless_control = False (bool)
  startup_homing = False (bool)
  enable_step_dir = False (bool)
  step_dir_always_on = False (bool)
  turns_per_step = 0.0009765625 (float)
  watchdog_timeout = 0.0 (float)
  enable_watchdog = False (bool)
  step_gpio_pin = 1 (int)
  dir_gpio_pin = 2 (int)
  calibration_lockin: ...
  sensorless_ramp: ...
  general_lockin: ...
  can_node_id = 0 (int)
  can_node_id_extended = False (bool)
  can_heartbeat_rate_ms = 100 (int)
fet_thermistor:
  error = 0x0000 (int)
  temperature = 25.999391555786133 (float)
  config: ...
motor_thermistor:
  error = 0x0000 (int)
  temperature = 0.0 (float)
  config: ...
motor:
  error = 0x0000 (int)
  armed_state = 0 (int)
  is_calibrated = True (bool)
  current_meas_phB = -0.15174412727355957 (float)
  current_meas_phC = 0.03545680642127991 (float)
  DC_calib_phB = -0.7747852206230164 (float)
  DC_calib_phC = -0.4790705740451813 (float)
  phase_current_rev_gain = 0.02500000037252903 (float)
  effective_current_lim = 10.0 (float)
  current_control: ...
  gate_driver: ...
  timing_log: ...
  config: ...
controller:
  error = 0x0000 (int)
  input_pos = 0.0 (float)
  input_vel = 0.0 (float)
  input_torque = 0.0 (float)
  pos_setpoint = 0.0 (float)
  vel_setpoint = 0.0 (float)
  torque_setpoint = 0.0 (float)
  trajectory_done = True (bool)
  vel_integrator_torque = 0.0 (float)
  anticogging_valid = False (bool)
  config: ...
  move_incremental(displacement: float, from_input_pos: bool)
  start_anticogging_calibration()
encoder:
  error = 0x0010 (int)
  is_ready = False (bool)
  index_found = False (bool)
  shadow_count = 0 (int)
  count_in_cpr = 0 (int)
  interpolation = 0.0 (float)
  phase = 0.0 (float)
  pos_estimate = 0.0 (float)
  pos_estimate_counts = 0.0 (float)
  pos_cpr = 0.0 (float)
  pos_cpr_counts = 0.0 (float)
  pos_circular = 0.0 (float)
  hall_state = 7 (int)
  vel_estimate = 0.0 (float)
  vel_estimate_counts = 0.0 (float)
  calib_scan_response = 0.0 (float)
  pos_abs = 0 (int)
  spi_error_rate = 0.0 (float)
  config: ...
  set_linear_count(count: int)
sensorless_estimator:
  error = 0x0000 (int)
  phase = -0.006432728376239538 (float)
  pll_pos = -0.005587339401245117 (float)
  vel_estimate = -0.17820142209529877 (float)
  config: ...
trap_traj:
  config: ...
min_endstop:
  endstop_state = False (bool)
  config: ...
max_endstop:
  endstop_state = False (bool)
  config: ...
watchdog_feed()
clear_errors()

In [2]: odrv0.axis0.motor
Out[2]:
error = 0x0000 (int)
armed_state = 0 (int)
is_calibrated = True (bool)
current_meas_phB = 0.2098262906074524 (float)
current_meas_phC = 0.1917402446269989 (float)
DC_calib_phB = -0.7738049030303955 (float)
DC_calib_phC = -0.4736144542694092 (float)
phase_current_rev_gain = 0.02500000037252903 (float)
effective_current_lim = 10.0 (float)
current_control:
  p_gain = 0.0326908677816391 (float)
  i_gain = 35.110958099365234 (float)
  v_current_control_integral_d = 0.0 (float)
  v_current_control_integral_q = 0.0 (float)
  Ibus = 0.0 (float)
  final_v_alpha = 0.0 (float)
  final_v_beta = 0.0 (float)
  Id_setpoint = 0.0 (float)
  Iq_setpoint = 0.0 (float)
  Iq_measured = 0.0 (float)
  Id_measured = 0.0 (float)
  I_measured_report_filter_k = 1.0 (float)
  max_allowed_current = 60.75 (float)
  overcurrent_trip_level = 67.5 (float)
  acim_rotor_flux = 0.0 (float)
  async_phase_vel = 0.0 (float)
  async_phase_offset = 0.0 (float)
gate_driver:
  drv_fault = 0 (int)
timing_log:
  general = 12786 (int)
  adc_cb_i = 2954 (int)
  adc_cb_dc = 12846 (int)
  meas_r = 1254 (int)
  meas_l = 22111 (int)
  enc_calib = 15916 (int)
  idx_search = 10114 (int)
  foc_voltage = 26988 (int)
  foc_current = 62703 (int)
  spi_start = 54814 (int)
  sample_now = 38212 (int)
  spi_end = 30942 (int)
config:
  pre_calibrated = True (bool)
  pole_pairs = 2 (int)
  calibration_current = 20.0 (float)
  resistance_calib_max_voltage = 2.0 (float)
  phase_inductance = 3.269086664658971e-05 (float)
  phase_resistance = 0.0351109579205513 (float)
  torque_constant = 0.03999999910593033 (float)
  direction = 1 (int)
  motor_type = 0 (int)
  current_lim = 20.0 (float)
  current_lim_margin = 8.0 (float)
  torque_lim = inf (float)
  inverter_temp_limit_lower = 100.0 (float)
  inverter_temp_limit_upper = 120.0 (float)
  requested_current_range = 60.0 (float)
  current_control_bandwidth = 1000.0 (float)
  acim_slip_velocity = 14.706000328063965 (float)
  acim_gain_min_flux = 10.0 (float)
  acim_autoflux_min_Id = 10.0 (float)
  acim_autoflux_enable = False (bool)
  acim_autoflux_attack_gain = 10.0 (float)
  acim_autoflux_decay_gain = 1.0 (float)

In [3]: odrv0.axis0.controller
Out[3]:
error = 0x0000 (int)
input_pos = 0.0 (float)
input_vel = 0.0 (float)
input_torque = 0.0 (float)
pos_setpoint = 0.0 (float)
vel_setpoint = 0.0 (float)
torque_setpoint = 0.0 (float)
trajectory_done = True (bool)
vel_integrator_torque = 0.0 (float)
anticogging_valid = False (bool)
config:
  gain_scheduling_width = 10.0 (float)
  enable_vel_limit = True (bool)
  enable_current_mode_vel_limit = True (bool)
  enable_gain_scheduling = False (bool)
  enable_overspeed_error = True (bool)
  control_mode = 2 (int)
  input_mode = 2 (int)
  pos_gain = 1.0 (float)
  vel_gain = 0.019999999552965164 (float)
  vel_integrator_gain = 0.10000000149011612 (float)
  vel_limit = 35000.0 (float)
  vel_limit_tolerance = 1.2000000476837158 (float)
  vel_ramp_rate = 2000.0 (float)
  torque_ramp_rate = 0.009999999776482582 (float)
  circular_setpoints = False (bool)
  circular_setpoint_range = 1.0 (float)
  homing_speed = 0.25 (float)
  inertia = 0.0 (float)
  axis_to_mirror = 255 (int)
  mirror_ratio = 1.0 (float)
  load_encoder_axis = 0 (int)
  input_filter_bandwidth = 2.0 (float)
  anticogging: ...
move_incremental(displacement: float, from_input_pos: bool)
start_anticogging_calibration()

In [4]: odrv0.axis0.encoder
Out[4]:
error = 0x0010 (int)
is_ready = False (bool)
index_found = False (bool)
shadow_count = 0 (int)
count_in_cpr = 0 (int)
interpolation = 0.0 (float)
phase = 0.0 (float)
pos_estimate = 0.0 (float)
pos_estimate_counts = 0.0 (float)
pos_cpr = 0.0 (float)
pos_cpr_counts = 0.0 (float)
pos_circular = 0.0 (float)
hall_state = 7 (int)
vel_estimate = 0.0 (float)
vel_estimate_counts = 0.0 (float)
calib_scan_response = 0.0 (float)
pos_abs = 0 (int)
spi_error_rate = 0.0 (float)
config:
  mode = 1 (int)
  use_index = False (bool)
  find_idx_on_lockin_only = False (bool)
  abs_spi_cs_gpio_pin = 1 (int)
  zero_count_on_find_idx = True (bool)
  cpr = 12 (int)
  offset = 33 (int)
  pre_calibrated = False (bool)
  offset_float = 0.5141406059265137 (float)
  enable_phase_interpolation = True (bool)
  bandwidth = 100.0 (float)
  calib_range = 0.02500000037252903 (float)
  calib_scan_distance = 50.26548385620117 (float)
  calib_scan_omega = 12.566370964050293 (float)
  idx_search_unidirectional = False (bool)
  ignore_illegal_hall_state = False (bool)
  sincos_gpio_pin_sin = 3 (int)
  sincos_gpio_pin_cos = 4 (int)
set_linear_count(count: int)

Any further help with this would be greatly appreciated.

Btw, have you upgraded to v0.5.1-rc4 as opposed to earlier release candidates? We found some bugs in the sensorless code. It takes commands in units of turns and turns/second now too.

No I have not. I will try that in the morning.

I got bored and tried 0.5.1-rc4 and I still cannot get above 14000 RPM.

In [14]: dump_errors(odrv0, True)
axis0
  axis: Error(s):
    ERROR_MOTOR_FAILED
  motor: Error(s):
    ERROR_CURRENT_UNSTABLE
  encoder: no error
  controller: no error

And now it wont even get past activating sensorless mode without throwing an unstable current error. It does the usual short spin up that it does when you activate sensorless, but then suddenly accelerates well over the programmed acceleration rate to what sounds to me like 20k RPM. Then it throws the unstable current error.

These CSV files were requested on discord for this issue

https://drive.google.com/file/d/1Y6Kj4ZNevrZqpxhu5KLmd3znzAgEOoMN/view?usp=sharing, https://drive.google.com/file/d/1cxr4VJHn7hVd8ZmI7geTIVEpVgqzdXkI/view?usp=sharing, https://drive.google.com/file/d/1igHApdG1JqnnIu2OMHgJzbRLsFxnZrtr/view?usp=sharing, https://drive.google.com/file/d/1uHs26gLKKc3F62pZt2vN5EXy8qpeOwSF/view?usp=sharing