~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Types of Tests and Testable Types

Use left/right arrows or swipe to navigate backwards/forwards

github.com/swoogles/TTTT

Why talk about this stuff?

• Unrelated Test Failures
• Long, complicated test cases
• Irreproducible failures

Non-Objectives

• Diving into any particular testing framework
• Convince you to do Test Driven Development
• Stop you from ever writing another Selenium test

Objectives

• Increase concern for seemingly innocent decisions in class design
• Fight back against exploding CI testing time
• Keep tests focused on the actual behavior of your class

Types of Tests

• Unit
• Boundary
• Integration (Interior)
• End-to-End
• Selenium
• Integration (Exterior)

Unit Tests

• Zero setup.
• Do not depend on any other implementations
• No periodic failures
• Screaming fast

Unit Tests

Boundary Tests

• Requires some setup
• Test one “neighboring” service
• Can introduce periodic failures
• Slower

Boundary Tests

Integration Tests - Internal

• More complicated to setup
• Depends on multiple implementations
• Multiple sources of periodic failures
• Multiple slow dependencies

Integration Tests - Internal

End-to-End Tests

• Maximally complicated to setup
• Requires all implementations in application
• Almost as slow as it gets
• Will confirm user expectations

End-to-End Tests

Selenium Tests

• More local environment dependencies
• Slow enough to watch them happen in real time!
• Still Necessary

Selenium Tests

Integration Tests - External

• You have no control over your dependencies
• Human-contact with another organization
• Is the other organization obligated to help?
• Clean explanation of production failures

Integration Tests - External

Code Coverage

How should we achieve full coverage?

Code Coverage

Code Coverage

Code Coverage

Code Coverage

Code Coverage

Code Coverage

Code Coverage

| Test Type | Coverage | | ——- | ——– | | Unit | Additive | | Integration | Combinatorial | | End-to-End | max(Combinatorial) | | Boundary | Additive |

Live Demo

• A generic Application with fallible components
• How service disruptions affect multiple suites
• Experience some intermittent failures.

Demo Concepts

class ServiceStatus {
    void ensureServiceIsRunning(String name) {
        // Check property file and throw a RuntimeException 
        // if turned off.
    }
}

Demo Concepts

abstract class UnreliableService {
    private List<UnreliableService> dependencies;
    
    Duration fallibleAction() {
        ServiceStatus.ensureServiceIsRunning(serviceName);
        this.dependencies.forEach(
            dependency->dependency.fallibleAction
        );
        ...
    }
}

Demo Concepts

UnreliableService Implementations

• Mapper
• Logic
• RestResource
• ThirdPartyResource
• Application

Demo Concepts

• TestSuiteCalculator
  • Organization - How many people?
  • CodeBase - How many classes?
  • TestingPeriod - How many executions?

Live Demo

Get thee to Intellij

Time-Sensitive Tests

Concepts that can be painful to model:

“Tomorrow”

“Last year”

“In the next 6 months”

Time-Sensitive Tests

We want tests that:

• Work today
• Work tomorrow
• Work after humanity has become multi-planetary

Time-Sensitive Tests

    void test() {
        List<Event> lastTen =
            eventLogic.inLast10Minutes(Instant.now());
        // Other commands...
        List<Event> lastOne = 
            eventLogic.inLastMinute(Instant.now());
            
        assertTrue( lastTen.containsAll(lastOne) );
    }

Entire Test runs at the current Instant

    void test() {
        Instant now = Instant.now();
        
        List<Event> lastTen = eventLogic.inLast10Minutes(now);
        // ... Other commands/assertions ...
        List<Event> lastOne = eventLogic.inLastMinute(now);
        
        assertTrue( lastTen.containsAll(lastOne) );
    }

Entire Test runs at a fixed Instant

    Clock clock = Clock.fixed(
                    Instant.parse("2018-08-08T00:00:00Z"),
                    ZoneId.systemDefault());
    void test() {
        Instant now = clock.instant();
        
        List<Event> lastTen = eventLogic.inLast10Minutes(now);
        // Other commands ...
        List<Event> lastOne = eventLogic.inLastMinute(now);
        
        assertTrue( lastTen.containsAll(lastOne) );
    }

Completely Time-Insensitive Test

    Clock clock = Clock.fixed(
                    Instant.parse("2018-08-08T00:00:00Z"),
                    ZoneId.of("America/Denver"));
    void test() {
        Instant now = clock.instant();
        
        List<Event> lastTen = eventLogic.inLast10Minutes(now);
        // Other commands ... 
        List<Event> lastOne = eventLogic.inLastMinute(now);
        
        assertTrue( lastTen.containsAll(lastOne) );
    }

Solo Project Stats

Mid-sized Organization Stats

Questions

Have any?