Good Setup, No Cleanup

You should always avoid creating tests that depends on results of preceding tests; thankfully this is the main purpose of DbUnit.

Don't be afraid to leave your trace after a test; principally with OneDatabasePerDeveloper. If you always put your database in a known state before a test execution, you usually don't need to clean it up. This simplifies your tests maintenance and reduces the overhead taken by the cleanup procedure. And sometimes, this is very helpful to manually verify the database state after executing a test that fails.