This is the case for me as well. I use the legacy body and aside from enabling BOM and picking nail colors for the first time, I only attach the HUD when I need to adjust feet poses. The few applier based clothing I use doesn't even need to go through the HUD except to enable or disable.