Interesting dilemma, my suggestion is to secure and manage your patient, transport in a timely fashion as per your protocols, and do one of two things:
If there is no one else there (in terms of residents) ask fire to watch the apartment form the outside and have LEO come and secure the scene as your patient was unable to secure it prior to transport.
If there are occupants in the apartment, leave a less than valuable piece of equipment behind, a penlight, cheap stethoscope etc., have a LEO return to the scene to retrieve this for you.
As medical personnel, we are not legally obligated to report these findings, and unless the drugs and paraphernalia have a pertinent part to play in the history they do not need to be included in the PCR. As human beings, I would suggest that we find a mechanism to at least alert Law Enforcement of your observations in such a way that you are not a direct party to that alert.