Why do buttons show an outline when clicked?

The outline is a focus indicator. It shows keyboard and assistive-technology users which control is active.

Sprocket Rocket buttons, HubSpot CTAs placed in a Sprocket Rocket module, and form submit buttons draw it as a ring in your theme's Primary color. It appears when the browser shows keyboard focus, such as when someone tabs to a button, and not after a mouse click. If it shows after a click, the installation has older theme files. Update the installation's files to get the current behavior. To change its color, change Primary in the theme's color settings. That also changes everything else that uses the Primary color.

It can be changed with custom CSS, but removing it without a visible replacement makes the page harder to use and can create an accessibility failure. Keep the default focus style or replace it with another clear indicator that meets your accessibility requirements.