Summary

Most subject line tests produce a winner that is really noise: too few sends, judged on inflated open rates, with the body changed at the same time. A fair test changes only the subject, judges on replies and waits for a sample that can tell the difference. Woodpecker lets you A/B test up to five versions. SpamCipher picks the winner for you on the metric you choose once every variant reaches the sample you set, and scores every subject for spam risk before it sends.

The subject line is the first thing a prospect sees and the easiest thing to test, so it is where most teams start A/B testing. It is also where most tests mislead. This guide covers how to run a subject line test that means something, and compares how Woodpecker and SpamCipher support it.

What Woodpecker offers

Woodpecker's pricing page, checked on October 26, 2026, lists A/B tests with up to 5 versions of an email, along with adaptive sending, a deliverability monitor, adjustable daily limits, customizable sending intervals and randomization. Its Email plan starts at $39 a month for 5,000 emails, priced by emails sent.

Five rules for a fair subject line test

  1. Change only the subject. Keep the body identical across variants, or you will not know which change won.
  2. Judge on replies. Open rates are inflated by privacy features that load images automatically. A subject that wins opens and loses replies did not win.
  3. Test two variants. More variants mean smaller samples for each.
  4. Wait for the sample. Decide the number of sends per variant before you start, and do not call the test early.
  5. Test real differences. "Quick question" against "quick question?" will never reach significance. A question against a statement, or a name against a problem, might.

How many sends a subject line test needs

ComparingSends per variant, roughly
Reply rate 2% vs 6%about 380
Reply rate 3% vs 5%about 1,500
Reply rate 4% vs 5%about 6,700
Open rate 40% vs 50%about 390

Figures are for 95% confidence and 80% power. Open-rate tests need fewer sends because the rates are higher, which is exactly why they are tempting, and exactly why inflated opens make them unreliable.

Side by side

NeedSpamCipherWoodpecker (from its pricing page)
VariantsUp to 5 per step, on every step of the sequenceA/B tests with up to 5 versions
Choosing the winnerAutomatic, on reply, click or open rate, once every variant reaches the sample you set (50 by default)Not listed on its pricing page
Subject riskSpam score on each step, covering the subject and the bodyNot listed on its pricing page
PlacementSeed tests by provider, including Gmail tabs and Outlook Focused or OtherDeliverability monitor
FormatHTML, first email in plain text, or all plain textNot listed on its pricing page

Subject line ideas worth testing

TestVariant AVariant B
Question vs statementHiring two SDRs?Your two new SDR mailboxes
Company vs problemIdea for {Company}Bounces after a list import
Short vs specificQuick ideaCutting your Gmail spam rate
Trigger vs no triggerAfter your Series AOutbound for a growing team

Keep subjects short, plain and honest. A fake "Re:" or "Fwd:" may lift opens once and costs trust and complaints every time after.

Test placement before you test subjects

A subject line test is meaningless if one variant lands in spam at Gmail and the other does not; you are measuring filtering, not wording. Run a placement test on each variant first. In SpamCipher, a campaign can be set to require a passing test before it launches.

Why SpamCipher for subject line testing

SpamCipher is an AI-powered sales engagement platform, B2B lead intelligence database, and email deliverability and security suite in one product, so a subject line is scored for spam risk, tested for placement and judged on replies in the same place, and sending protection keeps the test from costing you a mailbox. SpamCipher ONE is $94.99 a month ($75.99 billed annually). See sales engagement or start the 30-day free trial.

Category coverage: SpamCipher vs Woodpecker

SpamCipher is one platform across 11 market categories, all in one plan. The SpamCipher column shows what SpamCipher ONE includes in each category; the other columns show what each tool lists on its own website.

CategorySpamCipher ONEWoodpecker
Sales Engagement & Sales AutomationSequences with conditions, A/B variants, a CRM and booking pagesListed
B2B Lead IntelligenceSuper Search people and company database, and Email FinderPaid add-on (Lead Finder)
Email Deliverability Operations & Technical InfrastructureWarm-up on its own seed network, rotation, your own tracking domain, done-for-you domains and mailboxesListed
SDR EnablementUnibox, the Reply Agent, roles, assignment and a deal boardListed
Autonomous AI AgentsReply, Deliverability, Sales, Lead Finder and Signal agentsNot listed
AI Email Sequencing & Copy GenerationEmails from your profile, a 12-point check, sequences from approved emailsListed
AI-Driven Intent CategorizationEvery reply labelled by intent, custom labels and AI triageListed
Email Deliverability & TestingSeed placement tests by provider, scheduled tests, a spam score per stepNot listed
Email Marketing SolutionsVerification, bulk cleaning, ESP list hygiene with write-back, and an APIListed
Cybersecurity & Email Spam ProtectionDMARC aggregate and forensic reports, BIMI, MTA-STS, TLS-RPT and alertsNot listed
IT Monitoring & Uptime AnalyticsUptime checks every 5 minutes, domain health checks, blacklist monitoring, Postmaster dataNot listed
In one planAll 11 in SpamCipher ONE, $94.99 a month6 of 11, 1 more as paid add-ons

Competitor columns read from each vendor's own website on October 11, 2026 (Woodpecker). "Not listed" means we did not find it on their site.

Frequently asked questions

Be careful. Apple Mail Privacy Protection and image proxies load tracking pixels automatically, so open rates overstate real opens and add noise to any test. For cold email, reply rate is the safer judge, because a subject line that wins opens and loses replies has not won.
Two is usually right for cold email. Every extra variant splits the same list into smaller samples, and small samples cannot tell close subject lines apart.
Yes. Woodpecker's pricing page lists A/B tests with up to 5 versions of an email, and the subject line is one of the things a version can change.
Turn on automatic selection and pick reply, click or open rate. Each variant sends in rotation until every one reaches the number of sends you set, 50 by default, and the best then takes over. For subject lines with small expected differences, raise the number.
Yes. Spammy phrasing, all capitals, heavy punctuation or misleading prefixes such as a fake "Re:" can raise spam filtering and complaints. SpamCipher's spam score checks each step's subject as well as its body.

Run your whole outbound in one platform

Leads, sequences, replies, deliverability and email security, together. 30-day free trial. No credit card.

Start free trial