As we remain in uncertain C19 times, I took the opportunity to put together a new video-covering Sections D and E of the PCT in Oregon.
Stay safe everyone!
As we remain in uncertain C19 times, I took the opportunity to put together a new video-covering Sections D and E of the PCT in Oregon.
Stay safe everyone!
Issue:
Netbackup database has gone offline. All jobs are reporting a 2505 error. Online community recommends setting semaphores to official recommendations. backupserver already has the recommended settings in place.
Let’s dig into that.
View the current limits: sysctl -a | grep kernel.sem
kernel.sem = 300 307200 32 1024
View the current usage in a very messy manner: `ipcs
Count the currently used semaphores:
ipcs | grep Semaphore\ Arrays -A2000 | grep ^0 | wc -l (yes… there are cleaner ways of doing this- I am not making this pretty though)
This resulted in seeing 1024 currently used (out of 1024)
Stopping the DB resulted in 1015 consumed…
The thought at this point was that either netbackup was not releasing semaphores, or another process was consuming them. I hear they are tasty.
Let’s map semaphores to PIDs- this is a huge hassle to do manually as you have to do a lookup of each semaphore ID to a PID with the -s -i flags (specify semaphore with -s, specify -i for “print details on resource identified by id”)
Can anyone say “FOR LOOP”?
for pid in $( for semid in $( ipcs -s | awk ‘/0x/{ print $2 }’ ) ; do ipcs -s -i $pid | head -9 | tail -n1 | awk ‘{print $5}’ ; done
… this strips the key out of the results of ipcs -s, and then tosses the semaphore ID to ipcs via the specification of -i for a lookup on the specific semaphore. Easy.
We get this fallout: (with the highly sophisticated uniq -c tool…. Sarcasm is enhanced after 6 cups of coffee.)
3 2137
1 32550
5 2137
683 32550
1 36917
3 32550
281 36917
1 16356
6 36917
23 16356
1 6639
3 16356
We see a pattern here: pid 32550 has a bunch open!
So… what is that process?
Given that our max pid is 40960 on backupserver… we could have wrapped around again… (see: `cat /proc/sys/kernel/pid_max`) but if we are to believe what /var/log/messages has to say…
This was ‘nfsidmap[32550]’
The second-most semaphores loving PID was also nfsidmap: nfsidmap[36917]
A no-longer living process never released these. This was likely an issue with nfsidmap
So I rebooted backupserver- satisfied that the mine was still active in our environment… lurking for another day.
Conclusion:
Running backups again…
Only seeing 13 semaphores actively used:
[root@backupserver ~]# ipcs | grep Semaphore\ Arrays -A2000 | grep ^0 | wc -l
13
Two years late… and two years of hiking combined (due to miserable weather we dropped out and followed up the following year). Enjoy!
Happy Holidays everyone!
I have finally convinced myself to edit and publish my PCT video from this year (2016).
I hope you enjoy it!
After a bit of a delay, most of which was due to me needing to replace my computer… Here is my summary of CA Sections H and I:
The first destination was a lake called Thousand Island Lake, it was even more beautiful than I had remembered from my last visit here. The vomiting and passing out may have had an impact on my appreciation however. It is still one of the prettier lakes along the PCT so far.
Continuing up the 11,073ft Donohue pass and I came across this guy- frothing at the mouth as he envisioned eating my gear. I may have mentioned this before but I am not a big fan of Marmots. They are large mountain chipmunks that poo everywhere and eat through gear if you leave it unattended.
The route up to the pass was quite full of snow- something that I was just starting to realize would be the theme of this trip. This cup snow can be quite brutal on the ankles and it reflects the sun a little too well for my liking.
Finally at the top, the valley to Tuolumne Meadows can be seen in the distance. I had around 12 miles to do before 5pm so that I could buy some chocolate milk at the store. The route was fairly straightforward until I hit a cliff and had to backtrack a bit and head off to the left of the valley.
This was the end of the snow for the day, but the beginning of wet shoes. The Sierras had a bit of snow get dumped on them the week before I arrived, and then a heatwave went through which made the “seasonal streams” turn into raging rivers. It was as impressive as it was mood killing.
This was the Tuolumne River- something that I REALLY did not want to try to ford. There were bridges built in convenient locations which were appreciated. I was hurting fairly bad at this point due to chafing so I spent a bit of time at the Glen Aulen campground trying to avoid further damage. This is where I donned my swimsuit that I ended up wearing for the rest of the hike. Airflow is a good thing!
I bumped into my first (very) temporary hiking buddy named oldschool. I appreciated the company during the first few larger river crossings. This one was remarkably cold. Come to think of it, most of the river crossings were remarkably cold.
The next Pass was Benson Pass which sits at 10,150ft. The climb up was full of soft snow and the decent was the worst part of this trip. The trail turned into a constant stream which soaked my feet, and I ended up getting lost for an hour.
This is the decent right before the footprints disappeared and the cliffs appeared. Rock climbing is not what I like to do with snow covered ledges, but it seemed a bit safer than following the actual trail. I didn’t take any photos of the bad stuff as I was using my hands to keep myself steady.
After waking up early, I headed out from my camp and found some logs to use for the first couple of stream crossings. I love logs. I also ate some of this onion plant. These are quite spicy and add variation to my diet of poptarts and snickers bars.
The rest of the day consisted of wet, deep, and fast flowing stream crossings and steep snow fields. The trail just sucked at this point. I was surprised that I was still able to average just over 20 miles per day. The Sierras are certainly pretty, but you must pay your dues for the views.
I ended up camping with Little Cricket and Ducky who ended up being great hiking buddies. We hit the 1,000 mile marker at about 10am and took a few photos. It was nice to finally be done with that many miles of California, I still have quite a few more to hike before I finish that state unfortunately. Mosquitoes were bad here.
The rest of this trip was spent at fairly high altitude battling steep snow and a lot of rock fields. I am happy that I had company for this bit as it was frustratingly difficult and exposed. My mood wasn’t the greatest as my tendons in my feet were swollen and I had large blisters.
This trip was a good reminder of how much the trail can suck, and I called it quits as I needed 3 days to heal but didn’t have enough PTO to continue if I took that time off.
My 2014 hiking buddy Mr. President ended up saving the day massively and picked me up, gave me a place to stay, and got me a return flight the next day. I can’t thank him enough and look forward to hiking with him in the (hopefully near) future!
Two years ago, I dropped off the Pacific crest trail in a mountain town called Mammoth Lakes. I walked back south on the trail after becoming violently ill and took a bus to Reno. I had already decided that I would be dropping out after a couple more hundred miles but this illness cemented my decision.
So now I must finish the sections that I have left- and I am back in mammoth now.
This plane:
This plane:
And these free beers:
Got me here.
As an added bonus I was able to meet up with Big Red whom I met during my 2014 hike. He is a really nice guy who has kept up with the hiking community much better than I have.
Now all I have to do is get supplied and take a bus to the trailhead and hopefully walk a few hundred more miles!
I had the opportunity to take a longer weekend and visit Hawaii. This has been a bucket list item and I had a blast!
The hardest part for my planning was deciding what “visiting Hawaii” actually meant to me- and thankfully I had a travel buddy that helped broaden my scope of activities and enjoy the experience quite a bit more!
Overall- the plan was to get the most out of the least amount of time and I feel that we did a great job of hitting everything that we wanted- all while staying withing a decently strict budget. We flew into Kona and rented a car from Enterprise (I highly recommend spending a little more to go with a reputable car rental company). The overall thought process was to move about the island in a clockwise direction.
This resulted in a plan that included: