This question comes up from time to time and is worth careful consideration. Yes it is often more affordable due to increased volume of broadband facilities, especially in the US and the strong AUS dollar but it can be plagued by problems.

Hosting your website overseas does have some disadvantages.

Make sure you know what you are getting. What is the service like? The response time? The uptime? What software are they compatible with? Is there a language barrier? Do you forfeit fees if you are unhappy with the service? These options should be carefully considered.

The closer your server is to your clients, the faster those clients will be able to access your website. By hosting halfway across the globe, website response time for target demographics may be adversely affected.

There is also the issue of different time zones if the hosting services need to be contacted about your site, including the cost to call them OS. Do they have a 24/7 service centre and will they be able to respond quickly if your site goes down?

Search engines may also have trouble establishing who your 'target market' is and not rank you as relevantly for any 'local' business searches. Location of the server is one of the factors they use to determine location based results.
One area where it can help is if your target market is in a country different to your own. If you have your website running in your market location, this can help point search engines in the right direction of where you are targeting customers.

See this official video from Google - http://www.youtube.com/watch?v=keIzr3eWK8I

At a minimum you would have to use Google Webmaster tools to officially declare your targeted locations to help overcome this issue.

 At the end of the day is it worth a potential risk over the cost of hosting a domain name in Australia? Make sure you check out the pitfalls before diving in.