Hackerproof injection of secrets in kubernetes where none but your apps can use variables & secrets

 Okay, okay. It is not exactly hacker*proof* but definitely hacker resistant (you know like water resistant but not water proof). When we inject secrets into pods either via 

1. mounting, 
2. setting environment variables
3. using a helm values file with secrets and env vars
4. using CICD to inject while deployment
5. most tidious way: read from vault but this requires a role_id, secret_id of vault to be against set in pod as environment variable or some other way. Circular problem.

We stumbled upon a problem which actually ended up as a solution for our longterm question. How to hide secrets from a hacker or a rogue app which does ssh to the pod. If I am a hacker, I have ssh access to the pod, then I can see all secrets and environment variables. Can i set them up where only app can see it and nobody can?

example:

1. open up 2 shells
2. In shell 1 open your bashrc file, set some export or env vars and save.
3. Do `. /path/to/bashrc fiile' (notice the dot and a space after it)
4. In shell 1 you can access your new environment variable
5. In shell you cant or wont

example:

1. open up 2 shells
2. in shell one set an environment variable in the shell export xyz=abc and see if you can access it. you do.
3. In shell 2 you cant access the same.

Solution based on the above.

1. Mount secrets via vault or some place to some location in the pod /path/to/my/mount/vaultsecrets
2. In your entrypoint.sh file for your app, somewhere before you start your app, add `. /path/to/my/mount/vaultsecrets` 

Now when your pod on first boot, runs the entrypoint.sh, it sources all vars from your mount point and then starts your app in the same tty shell which means only this shell can access these environment variables and not the entire OS.

I had this secret injection employed with helm yaml file and I was itching for a more secure option to ensure nobody sees the secrets even if they connect to the pod but when we migrated from gitlab to github and github actions, my associate opted in this sidecar injection along with the helm yaml we had. It seems this unintended parallele secret injection via sidecar came as a blessing.

gotcha: So a hacker can literally manually run the command in the point no.2 and get the secrets?! Yes. but you can also add add another step after point. 2. Unmount that path. So now, you mounted the secrets, sourced it to a particular tty shell of the app and then unmounted it.