Free Software Testing Training Click For Info
Software Testing Training Is Coming To Your TownClick For Dates
Enter the Bug Free ZoneClick to Enter
Online Software Testing Training at your Pace and your PlaceClick For Details
Group Software Testing Training For Your TeamClick For Details
This course counts as two or three days Elective area of the Certified Test Manager (CTM) certification, Certified Software Test Professional certification and the CSTAS certification.
For costs and cities where this course might be offered, please see the right border of this page or to bring this course to your test team at your location, contact our Education and Professionals Services Group.
ConceptsThis workshop addresses the testing and quality concerns of real-time and embedded systems. It can be delivered in a generic form, or customized for a particular type of embedded system, such as an automated teller machine, automobile brake controller, router or other telecom device, gasoline pump, surgical tool, wristwatch, game console, etc., or focused for a particular regulatory agency (e.g., the U.S. FDA).
Real-time, embedded systems have a well deserved reputation for being difficult to test. Some of the complicating factors we examine in this workshop:
- Many real-time and embedded systems are high risk and safety critical, with human life dependencies.
- If your systems are safety critical, you may need to know how to apply relatively esoteric test methods like dynamic fault tree analysis and FMEA (failure mode effects analysis).
- The life dependencies mean that many of these systems fall within the purview of regulatory agencies such as the FAA, FDA, and NRC within the U.S.
- Many agencies conduct field audits which are widely dreaded, are staffed with people whose test competence is not respected, and publish inscrutable, ambiguous test regulations.
- Even when devices are not regulated nor safety-critical, they frequently have demanding specs that "push the envelope" in terms of response times, throughput volumes, error handling, etc.
- Many systems have significant limitations on their testability.
- Transient nature of test results - a test outcome may disappear before it can be captured (the "flicker effect").
- Speed - often, test tools and test environments are not fast enough to dynamically interact - they cannot keep up with the real-time devices.
- Real-time systems with their absolute deadlines may require test methods that are unknown in non-real-time systems, such as rate monotonic analysis, Markov chains and Petri nets.
- Accessibility and design for testability - typically, much of the internal behavior of an embedded system cannot easily be observed, and this hidden behavior is critical to understanding the device.
- Lack of conventional audit controls and audit trails (too much overhead).
- Resource sharing and thus possible interference - the system under test (SUT) and the testware must co-habitate, sharing the same processor and the same memory.
- The need to distribute testing, and to re-test across different development and execution platforms.
- Operational complexities
- Repairability - with the exception of expensive, specialized devices like FPGA chips, systems cannot be modified after burn-in. This has not infrequently lead to the recalls of millions of automobiles, for example, to replace defective devices. (With spacecraft, recalls are difficult, especially after the craft leaves our solar system.)
- Large numbers of execution and cross-development environments.
- Minimal internals that could help in testing: micro RTOS (real-time operating systems) are stripped down for speed and low resource consumption, with few of the capabilities of traditional OS that testers use; tight resources; and timing constraints on the delivery platform .
- The need to work reliably under extremes of temperature, radiation, and high g-forces (gravity, shaking), etc.
- Lack of test tools ; driving the need to build and maintain your own tools.
- Non-standard interfaces - e.g., you cannot use a Windows GUI tool to test an automobile brake controller.
Because of time limitations, not all these topics listed below may be addressed in a particular class or, if addressed, they may not all be examined in depth.
- REAL-TIME & EMBEDDED SYSTEMS - TERMS & CONCEPTS - This introductory section can be summarized, customized to the audience needs, or omitted altogether for experienced groups.
- QUALITY CONCEPTS & PRINCIPLES
- Basic Concepts
- Definitions of Testing
- Definitions of Quality
- Testing vs. Debugging
- Testing vs. Quality "Assurance"
- Testing vs. Modeling and Simulation
- Practical Limitations of Testing
- Risk Vs. Economics Trade-Offs
- Self-Assessment Exercise
- What's different from Conventional Testing
- Safety-Critical Systems: Testing Methods & Considerations
- Embedded Devices: Testing Methods & Considerations
- Real-Time Systems: Testing Methods & Considerations
- Real-Time Operating Systems (RTOS): Testing Considerations
- Basic Concepts
- DESIGNING TESTS - Part 1
- Generating Test Ideas
- What is a Test Case?
- Why do we need Methods for Identifying Test Cases?
- The Nature of Test Design
- Basic Test Case Design Techniques
- Functional Analysis
- Analyzing Functional Specs for Testability
- Input-Based Test Techniques
- Equivalence and Partitions
- Path Analysis
- Boundary Values
- Generating Test Ideas
- DESIGNING TESTS - Part 2
- Risk-Based Testing
- Exploratory Testing
- Real-Time and Embedded Test Methods
- Testing Memory and Memory Corruption
- Data Interference
- Testing the Timing
- Testing for Sequence Errors and Race Conditions
- Testing Timers and Testing for Timeouts
- Testing for Deadlocks and Live Locks
- Testing Counters
- Testing the Coordination of Concurrent Processes
- Semaphores, Mutex and Exclusion
- Testing Interrupts
- Testing Threads and Scheduling
- Test Planning
- The Importance of Test Planning
- Test Plan Template
- Test Planning in Regulated Industries
- Generic Overview, not Industry-Specific
- Industry- and Agency-Specific
- Major Issues of Test Planning
- Determining Test Objectives and Scope
- Allocated Effort among Types of Testing
- Planning Checkpoints, Milestones and Status Tracking
- System Quality Metrics
- Planning to Pass Regulatory Audits
- Testing Memory and Memory Corruption
- AUTOMATED TESTING
- Building and Using Test Harnesses
- Functions of Automated Test Tools
- Available Tool Examples (e.g., WATIR, Tet, Labware, etc.)
- Platforms
- Testing on the (non-Embedded) Development Platform
- Testing on other non-Embedded Platforms
- Testing on the Target (Real-Time and Embedded) Platform(s)
- Migrating Testware among Platforms
- Building Test Harnesses and Automation Frameworks
- Safety-Critical vs. Not
- Real-Time vs. Not
- Embedded vs. Not
- Real-time and Embedded System Testing
- Hardware-Software Co-Validation and Testing
- Mixed-Signal (A nalog and Digital) Testing
- Asynchronous vs. Synchronous Systems Testing
- BIST (Built-in Self Test)
- Design for Testability
- TEST EXECUTION & MANAGEMENT
- Major Issues of Test Execution
- Regulatory and Safety Concerns
- Criteria to Start Testing
- Driving the Tests
- Observability and Test Results Capture
- Capturing and Evaluating Outcomes
- Managing the Test Environment
- Problem Reporting and Tracking
- Re-Testing Fixes
- Criteria for Test Completion
- System & Acceptance Testing Success Factors
- Major Issues of Test Execution
- TESTING MODIFICATIONS
- Change and Version Control
- Change Completion Checklist
- The Four Levels of Defense
- Localized Change Testing
- Regression Testing
- Re-Test Coverage Guidelines
- Change and Version Control
- ENVIRONMENTAL TESTING
- Configuration & Compatibility Testing
- Calibration Testing
- Performance, Load and Stress Testing
- Security & Controls Testing
- Integration Testing
- Error and Disaster Recovery Testing
- Usability and User Scenario Testing
- Test Implications of System/Product Recall Policies
- Configuration & Compatibility Testing