Kyoto, Nara and Kobe is just an hour distance by train.
You can enjoy the rest of summer over there from Osaka.
Here is the information around Osaka.
Surrounded by a thick forest of massive cedars, the area known as Okuno-in,
The approach to get there, hundreds of stone statues are standing in line of both sides.
You will hear the sound of silence,when you walk there in the early morning.
You have a fun in Osaka then take some Zen meditation session at there to relax.
Koyasan Info http://www.shukubo.jp/eng/
Daimonji Okuribi in 16th Aug is the one of the seasonal tradition of Kyoto.→
Kyoto Info http://www.kyoto.travel/
Surrounded by the nature, North side of Kobe has historical hot spring town called "Arima Onsen".
You can walk in the woods or hike up to Mt, Rokko where can see the gorgeous view of Osaka and Kobe.
Take a bus from Osaka or a cable car from the Kobe city.
Arima Onsen Info http://www.arima-onsen.com/eng/index.html
Autumn season is also nice to see the color of leaves over there.