Yea I agree completely. Luckily nothing happened, but I probably should have been more assertive rather than simply stating that he had better be extremely careful and that there was no way in hell I was following him. I really didn't want to undermime him since he is almost 4 years older than me and has quite a bit of exploring under his belt. As a side note, I did mock him pretty heavily for wearing shorts.
Edit: I'd also like to point out that, although bold, he's a great guy and I'd trust him on any future explorations.