Iasi code camp 20 april 2013 marian chicu - database unit tests in the sql server world

Editor's Notes

• #2: "Unit testing is a well-understood concept in application development, but the database community has not yet embraced the advantages and strategies of this approach."This is a quote from an older MSDN whitepaper (march 2007) regarding unit testing in the SQL Server world. 6 years later I've found the situation is pretty similar, with many developers that write database code not really using this helpful methodology.
• #3: I miss you all!- 10 years experience (SQL Server 2000 -> 2k8 R2);- I like to learn & share technical info in the SO world (mostly on DBA.StackExchange.com)- don't tell my employer ;))! I do it only from home, really really!!- I'd like to see a SQL Server user group started (even if only for beers together, socializing should be mandatory in our field, we're so few);- Also a general database professionals group would be nice;Personal info:- I like reading about science stuff, IT security, psychology (Dan Ariely, PsyBlog);- Playing games on my PlayStation, playing poker, going to ski, airsoft;- as my friends already know, don't really like driving :-);
• #6: You know it is good for you, all the arguments make sense, so you start working out. There's an initial rush, which is great, but after a few days you start to wonder if it is worth the trouble. You're taking an hour out of your day to change your clothes and run on a hamster wheel and you're not sure you're really gaining anything other than sore legs and arms.Then, after maybe one or two weeks, just as the soreness is going away, a Big Deadline begins approaching. You need to spend every waking hour trying to get "useful" work done, so you cut out extraneous stuff, like going to the gym. You fall out of the habit, and by the time Big Deadline is over, you're back to square one. If you manage to make it back to the gym at all, you feel just as sore as you were the first time you went.
• #7: You know it is good for you, all the arguments make sense, so you start working out. There's an initial rush, which is great, but after a few days you start to wonder if it is worth the trouble. You're taking an hour out of your day to change your clothes and run on a hamster wheel and you're not sure you're really gaining anything other than sore legs and arms.Then, after maybe one or two weeks, just as the soreness is going away, a Big Deadline begins approaching. You need to spend every waking hour trying to get "useful" work done, so you cut out extraneous stuff, like going to the gym. You fall out of the habit, and by the time Big Deadline is over, you're back to square one. If you manage to make it back to the gym at all, you feel just as sore as you were the first time you went.
• #8: There's a misperception that testing belongs solely in the domain of QA. Developers are coders, and can't write tests.It's very hard to determine the value of test code.
• #9: Management doesn't really understand what unit testing really is, or why it has real intrinsic value to them.Management tends to be more concerned with rapid product delivery, and (incorrectly) sees unit testing as counterproductive to that goal.There's a common misperception that management will have to spend money to do unit testing correctly, despite the fact that the tools are freely available. (There is, of course, the developer ramp up time to consider, but it's not really prohibitive.)
• #10: XUnit is the family name given to bunch of testing frameworks that have become widely known amongst software developers. The name is a derivation of JUnit the first of these to be widely known. Siblings -> CppUnit, Nunit.The real root SUnit: Smalltalk community, Kent Beck was a big fan of automated testing at the heart of software development.The focus was on making it easy for programmers to define the tests using their regular Smalltalk environment, and then to run either a subset or a full set of tests quickly. Kent and his followers would run unit tests after every change to the system going through a rapid edit and test cycle in the Smalltalk IDE.
• #11: This is were a set of unit tests start (Test a Search stored procedure) -> several tests that fulfill this idea.
• #12: Test suite = (setup + teardown for the whole package) + tests (including their setup and teardowns)
• #13: Setup + teardown for the UNIT TEST
• #14: The setup() and teardown() methods serve to initialize and clean up test fixtures.
• #15: The setup() and teardown() methods serve to initialize and clean up test fixtures.
• #16: - bright individuals that get a new technology very fast (new languages, new platforms, new tools are acquired immediately as they appear);- in constant change (evolve continuously) - very rarely, mostly in big corporations, we see individuals that stay on the same technology for decades;- very good at writing complex database code (reports, searches, financial calculations);- sometimes want to avoid writing SQL code, they want to use an ORM (which is cool, unless you debug a 2 page query);- really a job of "Jack of all trades" (languages HTML, JS, C#, VB, CSS, ORMs, platforms for logging, for build, for source control, ..etc);- SQL is only a tool in their large handbag of tools;- think in objects or in procedures, butnot all the times get the "set way of thinking";
• #17: - don't care much about configuration, settings (integration testing -> files needed, settings that are changing from server to server -> connection strings, etc);- connections strings are ok?- files for input and output are existing?- want to make sure that functionality doesn't change (replace some LINQ with pure SQL and make sure the result is the same) -> transfer .net unit tests in db unit tests;- same approach as for the app code: a dev that's used to write tests for his .net code, will likely want to do the same for the SQL code;- sentiment of insurance when all unit tests pass (want to extend that to the db code, too);- feel the database as part of the project, another layer that's close, not a different area altogether (we now have source control, projects, builds for the databases) - they might see the db code as just as close and familiar as .net code (as long as technologies are similar and only the language is different);
• #18: - only in big companies;- there's no pure DBA job (in 99% of cases): sysadmin, SQL development, reports, BI implementations (ETL, aggregations);- usually they consider themselves the owner of the environment (which is not true):- everyone is owning the project (from the business owner, to the developer, QA, DevOps, DBA)- they do have the highest privileges on the db servers (but mostly because it's required, not because it's a playground);- like stability, because changes usually break things (deploys not really tested, system upgrades that don't go as expected (SP installation that breaks MS build and kills the server)) - lots of horror story especially in a 24/7 environment;- ugly ducklings (bad communication, no time, lack of insight in the app architecture, few people for many teams/environments/servers);- grumpy (see upper reasons);
• #19: They do performance tuning (which means REWRITING a lot of code)most of them don't write SQL code for the application (but help review it or make it faster);When wearing the developer hat, they share the benefits;what fails in the db is seen as their responsibility (and it's false, because the DBA maintains the whole environment, not individual pieces of code), so they don’t mind another safety net;unit tests would add another layer of safety among many others (eyeballing, manual executions, load tests, unit tests, QA env, acceptance, production). This might alleviate future pain.
• #20: - Represent the end user in the application development flow.-They do manual tests, automation, functionality, documentation, sometimes even talk to the customer to get facts.- sometimes even write code (load tests, automation of simple tests)- they have a hard life (sometimes one tester serves many developers, many apps, many environments, many clients)- interested in stability of the app;-they are the first real verification layer between the app and the real world;
• #21: - unit tests are a kind of contract between the dev and the app code that the code is stable;- piece of mind that easy cases are caught (flaws in the logic, bad data for input/output, some unnecessary delays are avoided) - if there is enough coverage;- don't see a single unit test, but see the whole, if there is at least 50% code coverage (and it’s for the complex stuff), or some useful number, they are happy and sure the developer is covering his back for simple and obvious cases;- avoid unnecessary clicks for future :-);
• #25: run fast, and make a difference: without them making changes and troubleshooting are more difficult.
• #26: run fast, and make a difference: without them making changes and troubleshooting are more difficult.
• #27: Nobody likes to wait
• #28: Nobody likes to wait
• #29: Without them making changes and troubleshooting are more difficult.
• #30: How do you test a new procedure?@debug parameter -> generate prints all over (check the flow of the procedure)Take all parameters and make sample callsCheck if it works for general cases
• #31: How do you test a new procedure?@debug parameter -> generate prints all over (check the flow of the procedure)Take all parameters and make sample callsCheck if it works for general cases
• #32: Unit tests find problems early in the development cycle.In test-driven development (TDD), which is frequently used in both Extreme Programming and Scrum, unit tests are created before the code itself is written. When the tests pass, that code is considered complete. The same unit tests are run against that function frequently as the larger code base is developed either as the code is changed or via an automated process with the build. If the unit tests fail, it is considered to be a bug either in the changed code or the tests themselves. The unit tests then allow the location of the fault or failure to be easily traced. Since the unit tests alert the development team of the problem before handing the code off to testers or clients, it is still early in the development process.
• #33: Unit testing allows the programmer to refactor code at a later date, and make sure the module still works correctly (e.g., in regression testing). The procedure is to write test cases for all functionsand methods so that whenever a change causes a fault, it can be quickly identified and fixed.Readily available unit tests make it easy for the programmer to check whether a piece of code is still working properly.In continuous unit testing environments, through the inherent practice of sustained maintenance, unit tests will continue to accurately reflect the intended use of the executable and code in the face of any change. Depending upon established development practices and unit test coverage, up-to-the-second accuracy can be maintained.
• #34: Unit testing may reduce uncertainty in the units themselves and can be used in a bottom-up testing style approach. By testing the parts of a program first and then testing the sum of its parts, integration testing becomes much easier.An elaborate hierarchy of unit tests does not equal integration testing. Integration with peripheral units should be included in integration tests, but not in unit tests.Integration testing typically still relies heavily on humans testing manually; high-level or global-scope testing can be difficult to automate, such that manual testing often appears faster and cheaper
• #35: Unit testing provides a sort of living documentation of the system. Developers looking to learn what functionality is provided by a unit and how to use it can look at the unit tests to gain a basic understanding of the unit's API.Unit test cases embody characteristics that are critical to the success of the unit. These characteristics can indicate appropriate/inappropriate use of a unit as well as negative behaviors that are to be trapped by the unit. A unit test case, in and of itself, documents these critical characteristics, although many software development environments do not rely solely upon code to document the product in development.By contrast, ordinary narrative documentation is more susceptible to drifting from the implementation of the program and will thus become outdated (e.g., design changes, feature creep, relaxed practices in keeping documents up-to-date).
• #36: A few obvious costs spring to mind:The time spent actually writing unit tests in the first placeNeed to define a suite of testsNeed to find out what to test/when to testNeed to configure a test environment / test configurationSearch for seed dataSearch for configuration files
• #37: Myth: the idea that unit testing inevitably lowers the cost of application development rests on the reasonable assumption that fixing a bug as soon as possible saves money. Graphs such as the one below typically compare two application’s costs to support this claim. Each solid black line represents an application’s total cost over time.
• #38: Real reason: They do not factor in the financial repercussions of a delay in the delivery of today’s enterprise applications.An organization’s profitability - the cost associated with the consequences of writing tests.Time expended coding unit tests keeps other features un-built and idling on backlog. The process of implementing unit tests cannot be readily scaled up to prevent backlog because, for most organizations, there are only one or two folks possessing the domain knowledge to add new or enhanced features to a specific application. When these developers code unit tests they are not coding new stuff.
• #39: A few obvious costs spring to mind:- The time spent actually writing unit tests in the first place- The time spent fixing and updating unit tests, either because you’ve deliberately refactored interfaces between code units or the responsibilities distributed among them, or because tests broke unexpectedly when you made other changes- The tendency – either by you or your colleagues – to avoid improving and refactoring application code out of fear that it may break a load of unit tests and hence incur extra work
• #40: A few obvious costs spring to mind:The time spent actually writing unit tests in the first placeNeed to define a suite of testsNeed to find out what to test/when to testNeed to configure a test environment / test configurationSearch for seed dataSearch for configuration files
• #41: - The time spent fixing and updating unit tests, either because you’ve deliberately refactored interfaces between code units or the responsibilities distributed among them, or because tests broke unexpectedly when you made other changesREFACTORING!Fixing broken unit tests!
• #42: - The tendency – either by you or your colleagues – to avoid improving and refactoring application code out of fear that it may break a load of unit tests and hence incur extra work
• #43: It makes you think before you write code.Guilty! Make all kind of assumptions -> start to code -> find problems -> avoid or ask questions -> lots of emails/skype/meetings -> deliver something that’s not very clearly documented!
• #44: It makes you think before you write code.So repeat after me: TDD is Thinking-driven development.
• #45: It makes you think before you write code.Sure, all the benefits are there: structured incremental progress, good coverage, and confidence when refactoring.But over and over again, when I see people starting to think about their first test, it comes back to me: It’s the think-before-you-act thing.In a user session from a Israel developers group (Israeli Software Craftsmanship User Group), we did the kata for counting code lines (3 times with 3 different groups). And three times, almost at the beginning, questions came up: -What constitutes a line? - In what language are we writing? - Do we count blank lines? - Does an “if” block counts as one line, even if it spreads over four lines? - Does a line need to contain compiling code?And so on. All of these questions came up before the first test (empty string, zero lines) or immediately after it. This kind of thinking and discussion before writing the code is what makes quality code.So repeat after me: TDD is Thinking-driven development.
• #47: Doesn’t have to be too complex to setup and configure (you have to just set it and forget it).Some early Microsoft test systems took almost a day to prepare (install all tools, all modules, prepare and configure everything).Developers didn’t really want to use them.
• #48: Difficult to obtain in a database environment. DBs are big.Possible solutions:Test configuration in Visual Studio projects -> different destination databaseSmall set of data (not 100 GBs of a db for testing functionality);Specific seed data for important tables (maybe few MB);This leads us to the next 2 questions: what to test and what NOT to test?
• #49: 1.A well written unit test works very well as such - it demonstrates how to run a module and shows the expected results;2. They need performance tuning more often, and when tuning we may have to completely rewrite a slow query, improving its performance while keeping its functionality intact.
• #50: 1. And when we fix those bugs, we need to make sure that we haven't introduce other ones;2. Because usually any unit tests involving the database run much slower than unit testing C++ or C# code alone, we typically do not add as many unit tests as we wish - we must keep the overall time the test harness runs low, within acceptable limits. So keeping unit tests for trivial modules may be very counterproductive as our systems grows, as we get more and more unit tests, and as we begin to wait too much until all our tests complete.
• #51: If our code refers a non-existing table or database, the database engine will throw a clear to understand error message. If this error message does not get stored in the error log, we should improve error handling. To verify that the DDL in our version control is in sync with the real thing, we can use tools such as SQL Compare.However, explicitly testing database modifications is rarely feasible: typically testing database modifications runs too slow, and the tests are quite delicate - we usually need to maintain them if the database schema changes. Suppose, for example, that we have added one more column to a table. If our select stored procedure does not use this new column, all existing unit tests invoking it can run as is. Unit tests verifying modifications of this table, however, usually need to change -  they need to either ignore this new column, or to include this new column in their expected results.
• #59: Reward the most talkative persons with:2 RedGate books2 packs of poker cards for Scrum estimationCDs of fine music from our sponsors
• #60: Ask them to review the speech, the content!