Plenty of good walking if that's your thing, the South Downs Way is accessible by bus intermediately at Birling Gap daily at this time of year if you don't want to go all the way to Seaford.
The Bluebell Railway is doable by public transport - from Eastbourne during the week the 270 from Haywards Heath serves Horsted Keynes Church Lane which is a short walk from the station. At weekends it serves HK station and on Saturdays from Lewes or Cooksbridge the 121 goes to Sheffield Park station and the gardens.
You can also get to Brighton for Volks Electric Railway on the seafront, and the chance of seeing a Thameslink train (or even a GWR 15x)
