Linux questions Rules (in addition of the Lemmy.zip rules)
Tips for giving and receiving help
Any rule violations will result in disciplinary actions
I love bash's -x option for exactly this type of debugging. It prints out every command before running it, so you can see what is different about the command as the script is running it
bash -x my-script.sh
Off the top of my head, I'm guessing the inline shell command $( ... ) is eating your double quotes, and I bet you should escape them, particularly around username and password
Thanks for the suggestion.
set -x didn't reveal any errors, and the commands that it'the script is executing are as I expect them to be.
I did just discover, though, a possible cause for my issue: https://lemmy.thewooskeys.com/comment/520854
what happens when you wrap $() in quotes?
Thanks for the suggestion. I just tried it and the result is the same. :(
Add echo before your curl command to see what the expanded command line actually is. Perhaps the arguments aren't as you expect them
Thanks for the suggestion. I already tried this: echo-ed the curl command to stdout and then in the shell copied it and pasted it to run it. From the script I get "000", from the shell I get "200".
I verified that the USERNAME, PASSWORD, and URL vars are being passed to the subshell.
But are they being passed into the script? Normal behavior would be for variables to propagate to subshells but not to scripts/commands, unless you explicitly export them (e.g. export USERNAME='my_username' instead of just USERNAME='my_username'). Is that what's happening here?
Another possibility is that when the script runs, it is run with a different shell, like sh or zsh. Does your script have a #! line at the top, and is it pointing to the same version of bash you are running in Terminal?
If you run echo $PATH; which curl in the script vs directly in your Terminal, do they output the same results? This is a bit of an edge case, but I've been stymied before by having multiple versions of curl installed (e.g. via homebrew and via my distro) and different PATHs in different contexts causing unexpected behavior.
Thanks for the suggestions.
The script is pulling the values of USERNAME and PASSWORD from a .env. I added debug echo $USERNAME $PASSWORD in the script and it shows the correct values, so the script is pulling the values correctly and storing them in the vars correctly. I also added that echo to the subshell command, like:
TEST=$(echo $USERNAME)
echo $TEST
...and the result was the correct USERNAME.
The script does begin with #!/bin/bash.
echo $PATH; which curl produces identical results when run from the shell and the script.
What exit code so you get from curl?
Ah!
TEST=$(curl -o /dev/null -s -k -w "%{http_code}" -u "${USERNAME}:${PASSWORD}" "${URL}/dashboard/#/http/routers")
echo "previous exit code: $?" #debug
echo "${TEST}" #debug
This outputs:
previous exit code: 7
000
Is $? referring to the exit command of the curl in the subshell? Or is it referring to the assignment of the subshell's output to the TEST var?
Exit code 7 means curl couldn't connect to the host, so I would try just curling a URL you know is valid directly, not setting it as an env var, to see what happens then.
So I guess the question is why is curl failing from within the subshell?
I stumbled on a possible cause, but more background is necessary to explain.
The script actually creates an ssh tunnel (to the Traefik host) and then does the curl. So the code is like:
SSH_CMD="ssh -N -L ${LOCAL_PORT}:127.0.0.1:${REMOTE_PORT} ${REMOTE_USER}@${REMOTE_HOST}"
$SSH_CMD &
SSH_PID=$!
SSH_RESULT=$?
TEST=$(curl -o /dev/null -s -k -w "%{http_code}" -u "${USERNAME}:${PASSWORD}" "${URL}/dashboard/")
echo "${TEST}" #debug
What I learned is that when i run the script, the tunnel is successfully created but the curl fails; but then if I run the script again a second tunnel is created and the curl works fine.
Ah, I see. ~~I guess they get different contexts or something?~~ (Edit: I re-read your post and this does not make any sense :)) What if you chain the ssh command and the curl using &&?
I'm not sure how to chain these 2 commands with &&, because the SSH command is being put in the background with &.
This doesn't work:
SSH_CMD="ssh -N -L ${LOCAL_PORT}:127.0.0.1:${REMOTE_PORT} ${REMOTE_USER}@${REMOTE_HOST}"
$SSH_CMD & && TEST=$(curl -o /dev/null -s -k -w "%{http_code}" -u "${USERNAME}:${PASSWORD}" "${URL}/dashboard/")
SSH_PID=$!
SSH_RESULT=$?
echo $TEST
Perhaps I don't need it in the background - the goal was to establish the tunnel and then continue with the script without it hanging until the ssh command is canceled.
Are you trying to reach a URL on the same host you're ssh-ing to? That would create some interesting effects.
Especially since it works the second time it could mean that the second time you're actually on the host and ssh-ing to the host itself and then curling localhost.
Try to add -f to ssh command
That seems to have done it!
Running the ssh -f... instead of ssh.... & seems to work first time and every time.
It makes it so SSH_PID=$! doesn't work, but I used pgrep -f <ssh command> instead.
Thanks!
You are very welcome! That was the hypothesis, that ssh doesn’t go into background as you want it to, since it works for the second run, but tunnel is there after the first.
Someone else suggested the env vars arent being expanded correctly inside the $(curl ...), which could be the culprit ... If a straight up URL works that would indicate that something like that is happening.
That said, I just tried setting an env var called URL="" and curling it, and curl said exit code 2, no URL specified, so something else is going on here.
changing how I called ssh in the background made the difference (https://lemmy.thewooskeys.com/comment/521006).
Thanks for your time and assistance
Here is the result of the script curling a known good URL (it still results in exit code 7 and thus a result of "000"), followed by a copy-paste of the curl command run in the shell (exited with "200"):
$ ./test.sh
curl -o /dev/null -s -k -w "%{http_code}" "https://i0.wp.com/www.notquitezen.co/wp-content/uploads/2022/07/Is-Happiness-an-Emotion.png"
previous exit code: 7
000
$
$ curl -o /dev/null -s -k -w "%{http_code}" "https://i0.wp.com/www.notquitezen.co/wp-content/uploads/2022/07/Is-Happiness-an-Emotion.png"
200