Case Study: Requesting Changes to GitAuto's Pull Requests

We've just released one of our most requested features - the ability to review and request changes to GitAuto's pull requests. Let's see it in action with a real-world example!

The Challenge

GitAuto's coverage dashboard detected that calculator.py in our sample-calculator repo had 0% test coverage. GitAuto automatically created a pull request to add tests. The initial test suite covered the basics well - but was it thorough enough?

First Attempt

Within minutes, GitAuto created a comprehensive test file covering all four operations (add, subtract, multiply, divide) plus the main function. Here's a snippet of the initial test structure:

test_calculator.py
+ class TestAdd:
+     def test_positive_numbers(self):
+         assert add(2, 3) == 5
+     def test_negative_numbers(self):
+         assert add(-1, -2) == -3
+     def test_zeros(self):
+         assert add(0, 0) == 0
+     def test_floats(self):
+         assert add(1.5, 2.5) == 4.0

GitAuto also covered error cases like division by zero:

test_calculator.py
+ class TestDivide:
+     def test_even_division(self):
+         assert divide(10, 2) == 5.0
+     def test_fractional_result(self):
+         assert divide(7, 2) == 3.5
+     def test_divide_by_zero_raises(self):
+         with pytest.raises(ValueError, match="Cannot divide by zero"):
+             divide(1, 0)

It even tested the interactive main() function using mocks for user input and print output. Solid start, but I noticed a missing edge case.

Requesting Changes: Repeating Decimals

While reviewing, I noticed the divide tests only covered clean divisions (10/2, 7/2). What about repeating decimals like 10/3? I left a review comment:

"test patterns you cant divide like 10 / 3 too"

GitAuto picked up the feedback and added the requested tests:

test_calculator.py
+     def test_repeating_decimal(self):
+         assert divide(10, 3) == pytest.approx(10 / 3)

It also added an integration test for the main function with repeating decimal input:

test_calculator.py
+     @patch("builtins.input", side_effect=["10", "/", "3"])
+     @patch("builtins.print")
+     def test_division_repeating_decimal_operation(self, mock_print, _mock_input):
+         main()
+         mock_print.assert_any_call(f"10.0 / 3.0 = {10.0 / 3.0}")

Notice how GitAuto used pytest.approx for the unit test to handle floating point comparison correctly, and tested the full end-to-end flow in the integration test. You can see the complete PR with all review comments.

For a step-by-step guide on setting up and using this feature, check out our Review Comment Trigger documentation.

The Power of Asynchronous Development

You might be thinking, "Wouldn't it be faster to just do this myself using Copilot, Cursor, or Windsurf in my IDE?" And you're not wrong! The models' capabilities are similar. However, this misses a crucial point.

Think of it this way: saying "I could do it faster myself" is like saying "I could do it faster than delegating to a team member." Of course you could - you know your repository better than anyone! But doing everything yourself isn't scalable.

The real power of GitAuto's approach is asynchronous development:

• You don't need to open your IDE
• You can review and provide feedback even from your phone
• While waiting for changes, you can review other pull requests by other team members or GitAuto

A Note on Tool Differences

As someone who uses both GitAuto and Cursor, I've noticed some key differences:

1. Cursor requires explicit file references - you'd need to provide calculator.py and test_calculator.py upfront in this case
2. In long conversations, Cursor sometimes loses track of the latest file contents for some reason
3. GitAuto refreshes its context with each interaction, leading to more consistent responses (and expensive for us actually... lol)

This isn't to say one tool is better - they serve different purposes. GitAuto optimizes for asynchronous workflows and team collaboration for software engineering managers and teams, while IDE-integrated tools excel at real-time coding assistance for individual software engineers.

The future of development isn't about replacing one tool with another - it's about knowing when to use each tool for maximum efficiency. Sometimes that means rapid local development with Copilot, and other times it means delegating to GitAuto while you focus on higher-level tasks.