I would say technically yes (with heavy emphasis on technically), but realistically no.
A google definition of urban exploration is:
the activity or pastime of exploring man-made structures, especially abandoned buildings and areas not generally open to the public. |
Streets are man-made structures so check off the first box.
But for "areas generally not open to the public", this is where things get tricky. Streets are typically open to the public but only if they are driving motor vehicles. Accessing streets via walking is technically illegal and not allowed. Thus when a person jaywalks across a street, they are accessing a man-made structure in a manner that is not open to people (aka walking instead of driving).
An example of this would be exploring subway tunnels. Using the proper transportation method of a subway car is perfectly allowed when traveling the subway system. However, attempting to access that same subway system via walking is very much not allowed. Jaywalking is like this but wayyy toned down.
So in conclusion, technically it is urban exploration. But realistically, you are just walking across a fucking street no it's not.
I can now take pride in the fact that I spent 15 minutes writing this ultimately useless post.