Are you conducting advertising based on gut feeling, without much planning, and perhaps not really knowing what works and what doesn’t? While this approach may yield okay results, it doesn’t effectively translate successes or failures into actionable insights.
You may have a hypothesis or a guess about the outcome of a test, but only by testing can you find out if it’s correct. For example, you may speculate that a question-based format in an ad text performs better than a declarative one. A/B testing is worthwhile so that you truly know what works, what doesn’t, and where results come from, enabling you to apply test learnings in the future.
In Aboad’s developed lean marketing process, the lean loop serves as the testing tool, allowing campaigns to be monitored at least once a week, sometimes daily.
A/B testing is an integral part of lean marketing
Lean marketing inherently involves developing processes and results through testing and learning. Marketing is executed so that every action (such as channel selection or advertisement) aligns with business objectives, and testing goals are predefined. Active testing has two clear benefits: it continuously optimizes advertising towards better results by identifying what works and what doesn’t while also accumulating valuable customer insights.
Since Aboad operates according to lean marketing principles, we conduct active A/B testing in almost all campaigns. In A/B testing, a variable is tested, interpreted, and monitored for selected outcomes. Such testing can be done directly in channels (e.g., in Facebook ad management or in the Experiments tool) or, as we do, manually using the lean loop. The latter method easily manages the process, results, and examination regardless of the channel or platform.
What should be considered in A/B testing?
To test one variable reliably, other variables should be as constant as possible. To ensure reliable results, notable considerations in testing include:
- Sufficient testing duration (at least two weeks, but results may take longer to accumulate).
- Control of other variables: if the test is conducted in summer or winter, results may vary significantly. Thus, testing should ideally be done under similar conditions, with the variable being the season.
- A large enough sample size (e.g., a sufficiently large target audience).
- Statistically significant sample (or at least a very clear difference).
How does the A/B testing process work in practice?
It’s crucial to define and clearly document the testing targets in advance and the ads in an ad planning file, for example. The accumulation of results must be actively monitored (at least weekly, more if necessary).
Lean loop assistance in testing
The lean loop is a document where tests are documented, insights can be tracked, and metrics (such as link CTR or ad-generated conversions) can be recorded.
In practice, the loop could be a Sheet file, with each test having its row and detailed campaign information such as:
- campaign status
- customer
- customer journey stage
- tactical action
- goal
- metric (KPI)
- channel
- campaign
- ad group
- micro-level testing phase
- tested item
- insights
- decisions for the future
- start date
- end date
- data update timing
- campaign owner
- other notes
- weekly notes if needed separately.
The lean loop also serves as a testing backlog, and information can easily be revisited in the file afterward. At Aboad, we use the lean loop as the basis for weekly client meetings and communicate development progress internally and externally.
What to test?
Testing should start from the top of the marketing funnel and progress downward as each stage becomes ready. Without any specific reasons to deviate, testing at the micro-level of marketing begins in the following order:
1. Target audience A/B testing at the channel level
- Once the best target audience is found, testing can proceed.
2. Creative content A/B testing
- Once the best creative content is found, testing can proceed.
3. Landing page optimization/testing
We’ll revisit specific testing topics and ideas later in our blog. By subscribing to the blog, you ensure you don’t miss future topics!
Tracking results
Results accumulation is monitored through suitable metrics in the channels. Based on these, observations are written either weekly or generally, and the insights gained from the test are recorded. Based on these insights, decisions can be made on what to do next and how to utilize the learnings.
With results, it’s essential to ensure that the sample size (n) is large enough (preferably over 1000). Obtaining results may require 3-10 rounds of A/B testing to be confident in the findings. Additionally, it may be reasonable to revisit previous test targets after a certain time; for instance, once the best audience is found, it’s worth retesting it periodically because effectiveness may be affected by evolving targeting possibilities in channels.
Using Data Studio for results analysis
Results can be easily tracked not only from the original file but also from a more visual Data Studio view, where information can be easily filtered and immediate insights, such as test durations as colored bars, can be seen. When the test duration is in red, it’s time to draw conclusions and develop further based on the test!
Testing and managing ad content angles become more challenging as AI takes over
Ad channels offer more targeting options and campaign types, heavily utilizing machine learning. Examples include Google Ads’ Performance Max or Facebook’s dynamic campaigns. The same trend applies to Google Ads search ads; when the ability to create expanded text ads was removed, AB testing became more difficult. However, you can still track the success of different combinations or place elements in locations to test content success. These campaign types simplify campaign management and may improve performance in some cases but also complicate testing by reducing marketers’ opportunities to influence campaign details.
Getting started with the lean loop
Schedule a discussion to see how the lean model could be utilized in your specific case.