How to Set Up Automated Tests with a QA Coding Agent for Flutter
Today, I'm sharing an interesting case study where a ride-sharing company (let's call them Company A) approached a system development vendor (Company B) for a mobile app revamp project. The requirement? Support both iOS and Android platforms while keeping costs reasonable. Company B's solution was to use Flutter with Dart for cross-platform development. To ensure quality delivery within the timeline, they wanted to implement automated testing to reduce manual testing efforts in the latter half of the project.
Let's see how we can leverage GitAuto, a QA Coding Agent, to set this up!
TL;DR - What's the story?
1. The Challenge
The first step was to set up GitHub Actions for automated testing. I created a GitHub issue with the title "Add a GitHub Actions workflow to automatically conduct flutter test". I only specified the trigger conditions in the description:
2. Initial Setup
Here is what GitAuto came up with - interestingly, it even provided a dummy test without being explicitly asked:
3. Verification
Let's check if the automated tests are actually running. Looking at the pull request checks:
All green! Let's also examine the test logs:
While the logs are minimal due to the dummy test, they confirm proper execution.
4. Analysis of GitAuto's Decisions
It's interesting to analyze how GitAuto handled ambiguous aspects that I didn't explicitly specify:
Default Branch Name
GitAuto correctly identified that this repository uses master instead of main - a detail I hadn't specified.
Flutter Version
GitAuto used Flutter 3.7.0, which is somewhat dated considering the latest version is 3.29.0 based on the official Flutter website.
Interestingly, Dart 3.7.0 is the latest version - perhaps there was some confusion there? In future updates, we should have GitAuto provide evidence for version selection decisions to save reviewers' time.
Actions/Checkout Version
GitAuto used v2, while v4 is the latest available according to the official GitHub Actions Checkout documentation. Another area for potential improvement.
Flutter Setup Approach
The choice of subosito/flutter-action was excellent. After investigation, I found it's a popular and well-maintained community action for Flutter, handling everything from installation to flutter pub get. As someone new to Flutter, this saved me significant research time.
You can specify your requirements if you don't rely on third-party actions to make sure GitAuto not use them.
5. Looking Forward
This experience was particularly valuable for me as a Flutter newcomer. It suggests that GitAuto can help set up testing infrastructure even for technologies we're not familiar with. In future posts, I'll explore how to leverage this setup for various types of Flutter tests, including:
- Unit tests
- Widget tests
- Integration tests
What's even more exciting is the potential democratization of test creation. With the CI/CD workflow in place, writing test cases could potentially be delegated to non-engineers or junior developers. We might even reach a point where technical leads and senior engineers can review and guide test development directly from their smartphones, without opening an IDE. This could fundamentally change how we think about engineering roles - one might start their journey directly from Tech Lead or Senior Software Engineer positions.
Feel free to share your thoughts and feedback here.