I've had a few discussions, regarding this and similiar subjects, with a friend of mine who is a prosecuting attorney for the District Attorney's office.
In a justified shooting, removing the components that are part of the series 80 system shouldn't be an issue. If you are lawfully exercising the use of deadly physical force, it doesn't matter what trigger pull weight is used, the type of sights on the gun, aftermarket grips, etc. The facts remain the same, to wit: your use of deadly physical force was reasonable, and necessary; you intended to use deadly physical force; and the gun discharged based on your intentional manipulation of the trigger.
What would matter is if the gun unintentionally discharges, or if your use of deadly physical force was unlawful. In these types of instances, then it could be claimed that your actions were reckless/careless.
Personally, if I am confident that the gun is safe to be carried/used without the Series 80 components, then I'd have no problem using it as a defensive weapon.