assertion-synthesizer

Assertion Synthesizer

Generate comprehensive test assertions by analyzing code implementation, behavior, and expected outputs.

Workflow

1. Analyze Code

Read and understand the implementation to identify testable behavior:

• Function signatures: Parameters, return types, side effects
• Logic paths: Conditionals, loops, branching behavior
• Edge cases: Boundary conditions, null/empty inputs, error conditions
• State changes: Object mutations, attribute modifications
• Dependencies: External calls, database operations, I/O

2. Identify Test Scenarios

Extract scenarios that need assertions:

• Happy path: Normal inputs producing expected outputs
• Edge cases: Empty lists, zero values, boundary conditions
• Error cases: Invalid inputs, exceptions, error handling
• State verification: Object state before/after operations
• Collection assertions: List contents, ordering, membership

3. Generate Assertions

Create appropriate assertions for each scenario:

Assertion Types:

• Equality: assert result == expected, assertEquals(expected, actual)
• Truthiness: assert is_valid, assertTrue(condition)
• Collections: assert item in list, assertContains(list, item)
• Exceptions: pytest.raises(Exception), assertThrows(Exception)
• State: assert obj.status == "active", assertEquals("active", obj.getStatus())

4. Structure Tests

Organize assertions into well-structured test functions:

• Use descriptive test names
• Follow Arrange-Act-Assert pattern
• Group related assertions
• Add explanatory comments

5. Verify Coverage

Ensure all important behaviors have assertions:

• Check all return values are tested
• Verify edge cases are covered
• Confirm error conditions are tested
• Validate state changes are asserted

Language-Specific Patterns

Python (pytest)

def test_basic_equality():
    # Arrange
    calculator = Calculator()

    # Act
    result = calculator.add(2, 3)

    # Assert
    assert result == 5

def test_collection_membership():
    items = get_active_items()
    assert "item1" in items
    assert len(items) == 3

def test_exception_handling():
    with pytest.raises(ValueError, match="Invalid input"):
        process_data(None)

def test_state_change():
    user = User("Alice")
    user.activate()
    assert user.is_active == True
    assert user.status == "active"

Python (unittest)

def test_basic_equality(self):
    calculator = Calculator()
    result = calculator.add(2, 3)
    self.assertEqual(result, 5)

def test_collection_membership(self):
    items = get_active_items()
    self.assertIn("item1", items)
    self.assertEqual(len(items), 3)

def test_exception_handling(self):
    with self.assertRaises(ValueError):
        process_data(None)

def test_state_change(self):
    user = User("Alice")
    user.activate()
    self.assertTrue(user.is_active)
    self.assertEqual(user.status, "active")

Java (JUnit + AssertJ)

@Test
void testBasicEquality() {
    Calculator calculator = new Calculator();
    int result = calculator.add(2, 3);
    assertThat(result).isEqualTo(5);
}

@Test
void testCollectionMembership() {
    List<String> items = getActiveItems();
    assertThat(items).contains("item1");
    assertThat(items).hasSize(3);
}

@Test
void testExceptionHandling() {
    assertThatThrownBy(() -> processData(null))
        .isInstanceOf(IllegalArgumentException.class)
        .hasMessage("Invalid input");
}

@Test
void testStateChange() {
    User user = new User("Alice");
    user.activate();
    assertThat(user.isActive()).isTrue();
    assertThat(user.getStatus()).isEqualTo("active");
}

JavaScript/TypeScript (Jest)

test('basic equality', () => {
    const calculator = new Calculator();
    const result = calculator.add(2, 3);
    expect(result).toBe(5);
});

test('collection membership', () => {
    const items = getActiveItems();
    expect(items).toContain('item1');
    expect(items).toHaveLength(3);
});

test('exception handling', () => {
    expect(() => processData(null))
        .toThrow('Invalid input');
});

test('state change', () => {
    const user = new User('Alice');
    user.activate();
    expect(user.isActive).toBe(true);
    expect(user.status).toBe('active');
});

Assertion Synthesis Strategies

From Return Values

Analyze what the function returns and assert on it:

# Code:
def get_full_name(first, last):
    return f"{first} {last}"

# Synthesized assertion:
def test_get_full_name():
    result = get_full_name("John", "Doe")
    assert result == "John Doe"
    assert isinstance(result, str)

From Side Effects

Identify state changes and mutations:

# Code:
def withdraw(account, amount):
    if account.balance >= amount:
        account.balance -= amount
        return True
    return False

# Synthesized assertions:
def test_withdraw_success():
    account = Account(balance=100)
    result = withdraw(account, 50)
    assert result == True
    assert account.balance == 50

def test_withdraw_insufficient_funds():
    account = Account(balance=30)
    result = withdraw(account, 50)
    assert result == False
    assert account.balance == 30  # Balance unchanged

From Conditional Logic

Test each branch:

# Code:
def classify_age(age):
    if age < 0:
        raise ValueError("Invalid age")
    elif age < 18:
        return "minor"
    elif age < 65:
        return "adult"
    else:
        return "senior"

# Synthesized assertions:
def test_classify_age_invalid():
    with pytest.raises(ValueError):
        classify_age(-1)

def test_classify_age_minor():
    assert classify_age(10) == "minor"
    assert classify_age(17) == "minor"

def test_classify_age_adult():
    assert classify_age(18) == "adult"
    assert classify_age(64) == "adult"

def test_classify_age_senior():
    assert classify_age(65) == "senior"
    assert classify_age(100) == "senior"

From Collections

Assert on collection properties:

# Code:
def filter_active_users(users):
    return [u for u in users if u.is_active]

# Synthesized assertions:
def test_filter_active_users():
    users = [
        User("Alice", is_active=True),
        User("Bob", is_active=False),
        User("Charlie", is_active=True)
    ]
    result = filter_active_users(users)

    assert len(result) == 2
    assert all(u.is_active for u in result)
    assert result[0].name == "Alice"
    assert result[1].name == "Charlie"

From Object State

Verify object properties:

# Code:
class ShoppingCart:
    def __init__(self):
        self.items = []
        self.total = 0

    def add_item(self, item, price):
        self.items.append(item)
        self.total += price

# Synthesized assertions:
def test_shopping_cart_add_item():
    cart = ShoppingCart()

    # Initial state
    assert cart.items == []
    assert cart.total == 0

    # After first item
    cart.add_item("book", 20)
    assert len(cart.items) == 1
    assert "book" in cart.items
    assert cart.total == 20

    # After second item
    cart.add_item("pen", 5)
    assert len(cart.items) == 2
    assert cart.total == 25

Best Practices

Assertion Quality

• Be specific: Prefer assert result == 5 over assert result > 0
• Test one thing: Each test should verify one specific behavior
• Use meaningful values: Avoid magic numbers, use named constants
• Check types: Verify return types when relevant

Coverage Completeness

• All branches: Test every if/else path
• Boundary values: Test min, max, zero, empty, null
• Error cases: Assert on exceptions and error messages
• State transitions: Verify before/after states

Test Organization

• Descriptive names: test_withdraw_insufficient_funds not test_case_2
• Arrange-Act-Assert: Clear separation of setup, execution, verification
• Group related tests: Use test classes or describe blocks
• Comment complex assertions: Explain what's being verified

Common Patterns

Multiple assertions for comprehensive verification:

def test_user_registration():
    user = register_user("alice@example.com", "password123")

    # Verify all important properties
    assert user.email == "alice@example.com"
    assert user.is_active == False  # Not activated yet
    assert user.created_at is not None
    assert user.id is not None
    assert len(user.roles) == 1
    assert "user" in user.roles

Parametrized tests for multiple inputs:

@pytest.mark.parametrize("input,expected", [
    (0, "zero"),
    (1, "one"),
    (5, "many"),
    (100, "many"),
])
def test_number_to_word(input, expected):
    assert number_to_word(input) == expected

Example Sessions

Example 1: Untested Function

User provides code:

def calculate_discount(price, discount_percent):
    if discount_percent < 0 or discount_percent > 100:
        raise ValueError("Discount must be between 0 and 100")
    return price * (1 - discount_percent / 100)

Synthesized assertions:

def test_calculate_discount_normal():
    assert calculate_discount(100, 10) == 90.0
    assert calculate_discount(50, 20) == 40.0

def test_calculate_discount_zero():
    assert calculate_discount(100, 0) == 100.0

def test_calculate_discount_full():
    assert calculate_discount(100, 100) == 0.0

def test_calculate_discount_invalid_negative():
    with pytest.raises(ValueError, match="between 0 and 100"):
        calculate_discount(100, -10)

def test_calculate_discount_invalid_over_100():
    with pytest.raises(ValueError, match="between 0 and 100"):
        calculate_discount(100, 150)

Example 2: Enhancing Existing Tests

User provides weak test:

def test_sort_list():
    result = sort_list([3, 1, 2])
    assert result  # Only checks if result exists

Enhanced assertions:

def test_sort_list():
    result = sort_list([3, 1, 2])

    # Verify actual sorting behavior
    assert result == [1, 2, 3]
    assert len(result) == 3
    assert isinstance(result, list)

def test_sort_list_empty():
    assert sort_list([]) == []

def test_sort_list_already_sorted():
    assert sort_list([1, 2, 3]) == [1, 2, 3]

def test_sort_list_duplicates():
    assert sort_list([3, 1, 2, 1]) == [1, 1, 2, 3]

Tips

Analyzing Complex Code

• Break down complex functions into testable units
• Identify all execution paths through code
• Look for implicit assumptions to test
• Consider edge cases the code may not handle

Capturing Behavior

• Run code mentally or actually execute it
• Note all observable outputs and state changes
• Document expected behavior in assertions
• Test both success and failure scenarios

Balancing Coverage

• Prioritize critical paths over trivial code
• Don't over-assert on implementation details
• Focus on public API behavior
• Test the contract, not the implementation