One annoying problem I've noticed in the past is that if an exception is thrown in Application_Start() the application does not restart, resulting in subsequent requests going through to the application (aspx pages and other handlers) without the Application_Start() actions happening. This can be particularly annoying as the errors that result on the pages of the site do not tell you why Application_Start threw an exception to begin with.
Luckily there is a simple solution, and it involves using HttpRuntime.UnloadAppDomain. If you call UnloadAppDomain the application will reset and the next request will start it up again, causing Application_Start to occur again. This will allow you diagnose the error that is occuring in Application_Start and thus let you fix the problem.
Below is a sample Application_Start method: