SoftwareManCal

Organization in test files is an interesting topic, theres the organization alphabetically, by contexts, or theres just none and we put all new tests at the bottom. I draw the comparison to moving; when you first get all your boxes in your new place, you have to start somewhere. You start unpacking boxes and just putting stuff somewhere, it doesnt quite matter where because you know youll reorganize it later. You just have to put it somewhere so you can move on and quite literally create calculated tech debt to be address later (supposedly). This is the case for most test suites. Just put the code in there, we’ll organize or optimize it later, or so we say.

As time goes on, we may tidy up and organize some aspects, but the more content we pile into a file, the trickier it gets to keep things organized. Files that are rarely touched tend to stay messy since no one bothers to optimize them. However, if we plan ahead and maintain a consistent organizational structure throughout our testing suite, we can set ourselves up for easier optimization down the road, especially as files begin to bloat.

Navigating through spec files that span thousands of lines can be quite a challenge. Whether you’re adding tests for new code or prefer writing tests before diving into the code (depending on your developer style), figuring out where to insert the code can become a daunting task. As you scroll through those lengthy files, everything starts to blur together, making it hard to locate specific sections like class methods, scope methods, or instance methods. This confusion increases the risk of inadvertently duplicating tests because you might miss existing ones while scrolling through the file. How do you ensure you’re not doubling up on tests that are already there? It’s a real puzzle.

The Basics

Thats where our basic organization blocks come into play. When looking at a model spec file, we really should have about 6 blocks, theres always little more or a less, but when we think of our ideal state in a Model file, I would expect to see a few of these.

RSpec.describe PostSerializer, type: :serializer do
  describe "associations" do
  end
  
  describe "validations" do
  end
  
  describe "callbacks" do
  end
  
  describe "scopes" do
  end
  
  describe "class methods" do
  end
  
  describe "instance methods" do
  end
end

Ill be using a real world example and cannot share the code, but its also incredibly long files (so you wouldn’t want to read them anyway.)

The User Model Spec

In this example we’re looking at the User Model spec file, it currently sits at 3500 lines, and after 5 runs, we have an average test time of 81.67 seconds to run 381 examples and file load times of 10.66 seconds. Looking at the factory creation, we create a total of 1578 objects to run these tests.

[TEST PROF INFO] Factories usage

Total: 1578
Total top-level: 695
Total time: 00:51.531 (out of 01:07.222)
Total uniq factories: 29

This file is one of the older ones in our codebase, and unfortunately, it’s a bit of a mess. There are thousands of factories being created, tests that might be duplicated or not organized properly, and some tests that haven’t been grouped together correctly. While we do have some organized blocks like callbacks, validations, and associations, not everyone has stuck to this structure throughout the file. For instance, you might find validations outside their designated blocks or methods placed in the wrong sections.

At the bottom of the file, there’s a comment block labeled “deprecated tests” that has been sitting there for five years, despite new tests for recent features being added just days ago. This kind of neglect can really weigh down a test file. When new developers come in, they hesitate to clean things up because they lack context about what each block means. Plus, they’re unsure if someone has added new tests below that “deprecated tests” comment. It’s challenging for them to dive into the file and figure out where everything should go.

If we were to organize this file by the type of test, it would be much clearer and easier to understand for new developers.

Starting With One Section: User Scopes

Let’s focus on refining the User Scopes. After isolating them, we found a batch of 39 tests that all pass within approximately 9 seconds, involving the creation of 138 factories. Our next steps involve reviewing these tests, looking for opportunities to combine them, identifying any missing spec coverage, and considering conversions to ‘let_it_be’s (using the ruby-prof gem).

Test Aggregation

Starting with test aggregation alone, we managed to trim down 9 tests and reduce our factories to 104, resulting in an overall test time of around 5 seconds. This minor effort already slashed our testing time nearly in half! The significance of test aggregation, especially with scoping, becomes evident as each ‘it’ block recreates the user and its associations. Many scoping instances involve unnecessary objects in the database to validate the scoping. For instance, multiple active_user and inactive_user instances might be built for each test, adding considerable time due to cascade effects.

Removing the Redundancy

With tests now aggregated, we can identify redundant factories and eliminate them. It’s common to recreate the same factory multiple times, and reusing them instead of building new ones further accelerates our testing. During this process, we unexpectedly discovered duplicate scope tests that were nearly identical. These duplicates, previously buried at the bottom of the file instead of within the scope section, allowed us to remove 5 examples, reducing factories to 80 and test time to 4.25 seconds.

Particularly with scopes, there’s a tendency to create excessive users to prove a point. Simplifying by using just one active and one inactive user can often suffice to validate the test. Continuing through the file, we eliminated more factories, ensuring we don’t oversimplify tests but also avoid unnecessary complexity.

After these optimizations—cleaning up duplicates, organizing blocks, minimizing object creation, aggregating tests, and implementing ‘let_it_be’s—we scrutinized our object creation and removed redundant objects used solely for attribute assignments. For example, three top-level users initially created were later overwritten with the same names in lower blocks, allowing us to update the originals instead of creating new factories with extra associations.

The Payoff

The result? An average test time of 3.14 seconds, just 42 factories created, and a streamlined test file reduced to only 23 while maintaining full coverage. This overhaul resulted in a remarkable reduction in test time by 65.87%, factories cut by 69.57%, and tests decreased by 41.03%. All accomplished within a total development time of 2 hours

[TEST PROF INFO] Factories usage

 Total: 42
 Total top-level: 24
 Total time: 00:02.354 (out of 00:09.134)
 Total uniq factories: 7

One of the notable benefits of these optimizations is our ability to swiftly compare the model file with the scope spec file, identifying any scopes that might be lacking in testing. Additionally, maintaining this file has become much more manageable. We can confidently say there are no duplicate tests, and although there’s always room for improvement, the file’s performance has significantly improved.

I understand that without seeing the actual file, it might be challenging to fully grasp the transformations. However, as you deconstruct a file and examine its components, patterns emerge, revealing the extent of unnecessary objects and duplication. Large files, especially those exceeding 3000 lines, can feel overwhelming, but breaking them down incrementally and running tests after each change helps preserve your sanity and brings those patterns to light.