Timeline of Activities Leading to the Certification of the Boeing 737 MAX 8 Aircraft and Actions Taken After

Timeline of Activities Leading to the

Certification of the Boeing 737 MAX 8

Aircraft and Actions Taken After

the October 2018 Lion Air Accident

Report No. AV2020037

June 29, 2020

Timeline of Activities Leading to the Certification of the Boeing

737 MAX 8 Aircraft and Actions Taken After the October 2018

Lion Air Accident

Requested by the Secretary of Transportation;

the Chairmen of the House Committee on Transportation and

Infrastructure and its Subcommittee on Aviation; the Chairman and Ranking Member of the Senate Committee

on Appropriations, Subcommittee on Transportation, Housing and Urban Development, and Related Agencies;

and Senat

or Richard Blumenthal

Federal Aviation Administration

| AV2020037 | June 29, 2020

What We Looked At

The Federal Aviation Administration (FAA) is responsible for the safety and certification of all civilian aircraft

manufactured and operated in the United States. However, two accidents in late 2018 and early 2019 involving

Boeing 737 MAX 8 aircraft raised significant safety concerns about FAA’s certification of this aircraft. On March

19, 2019, Secretary of Transportation Elaine L. Chao requested that we compile an objective and detailed

factual history of the activities that resulted in the certification of the 737 MAX 8. We also received similar

requests from the Chairmen of the House Committee on Transportation and Infrastructure and its

Subcommittee on Aviation; the Chairman and Ranking Member of the Senate Committee on Appropriations,

Subcommittee on Transportation, Housing and Urban Development, and Related Agencies; and Senator

Richard Blumenthal. They requested that we review aspects of FAA’s approach to certifying the MAX series of

aircraft, its reliance on the Organization Designation Authorization (ODA) program, and the Agency’s actions

following the two accidents. Our overall audit objective was to determine and evaluate FAA’s process for

certifying the Boeing 737 MAX series of aircraft.

What We Found

In this report, we provide a detailed timeline of the activities resulting in the certification of the 737 MAX 8,

beginning in January 2012, when Boeing submitted its initial application for an Amended Type Certificate to

FAA. This report also compiles a timeline of events following the October 29, 2018, crash of Lion Air Flight 610

up until the crash of Ethiopian Air Flight 302 on March 10, 2019. In addition, during the same time period as

FAA’s certification efforts, Boeing, FAA, and our office were identifying issues that—although not specific to the

737 MAX 8—may have impacted the original certification of the aircraft. As such, we also provided a timeline of

concurrent related oversight actions and events related to FAA’s ODA program.

Our Recommendations

We are not making recommendations in this report. The data gathered are informational and represent our

observations in response to the Secretary’s and other congressional requests. We will report further on FAA’s

oversight of the certification process and other related matters, as well as make recommendations as

applicable, in future reports.

All OIG audit reports are available on our website at www.oig.dot.gov.

For inquiries about this report, please contact our Office of Government and Public Affairs at (202) 366-8751.

AV2020037

Contents

Memorandum 1

Background 3

Results in Brief 8

Timeline of Activities Leading to the Certification of the Boeing 737 MAX 8

Aircraft 10

Timeline of Events Between the Lion Air and Ethiopian Airlines Crashes 29

Timeline of Concurrent FAA Organization Designation Authorization

Oversight Actions and Events 34

Conclusion 38

Recommendations 38

Agency Comments and OIG Response 38

Exhibit A. Scope and Methodology 39

Exhibit B. Organizations Visited or Contacted 41

Exhibit C. Glossary of Terms 42

Exhibit D. Major Contributors to This Report 45

Appendix. Agency Comments 46

AV2020037 1

U.S. DEPARTMENT OF TRANSPORTATION

OFFICE OF INSPECTOR GENERAL

Memorandum

Date: June 29, 2020

Subject: INFORMATION: Timeline of Activities Leading to the Certification of the Boeing

737 MAX 8 Aircraft and Actions Taken After the October 2018 Lion Air Accident

Report No. AV2020037

From: Howard R. “Skip” Elliott

Acting Inspector General

To: Federal Aviation Administrator

The Federal Aviation Administration (FAA) is responsible for the safety and

certification of all civilian aircraft manufactured and operated in the United

States. While FAA has an excellent safety record, two accidents in late 2018 and

early 2019 involving Boeing 737 MAX 8

aircraft have raised significant safety

concerns about FAA’s certification of this aircraft.

On October 29, 2018, Lion Air Flight 610 crashed into the Java Sea shortly after

departing Soekarno-Hatt International Airport, Jakarta, resulting in 189 fatalities.

Just over 4 months later, on March 10, 2019, Ethiopian Air Flight 302 crashed

shortly after departing Addis Ababa Bole International Airport, resulting in

157 fatalities, including 8 Americans.

On March 19, 2019, Secretary of Transportation Elaine L. Chao requested that we

compile an objective and detailed factual history of the activities that resulted in

the certification of the 737 MAX 8. We also received similar requests from the

Chairmen of the House Committee on Transportation and Infrastructure and its

Subcommittee on Aviation; the Chairman and Ranking Member of the Senate

Committee on Appropriations, Subcommittee on Transportation, Housing and

Urban Development, and Related Agencies; and Senator Richard Blumenthal.

They requested that we review aspects of FAA’s approach to certifying the MAX

series of aircraft, its reliance on the Organization Designation Authorization

(ODA) program,

and the Agency’s actions following each of the two accidents.

The official model number of the Boeing 737 MAX 8 is the 737-8.

FAA created the ODA program in 2005 to standardize its oversight of organizational designees (e.g., aircraft

manufacturers) that have been approved to perform certain functions on the Agency’s behalf, such as determining

compliance with aircraft certification regulations.

AV2020037 2

This is the first report that the Office of Inspector General (OIG) is providing

related to these requests. Our overall audit objective was to determine and

evaluate FAA’s process for certifying the Boeing 737 MAX series of aircraft. In this

report, in response to the Secretary’s request, we provide a detailed timeline of

the activities resulting in the certification of the 737 MAX 8. In addition, in

response to multiple congressional requests, this report includes timelines of

events following the October 2018 Lion Air crash up until the March 2019

Ethiopian Air crash and concurrent related oversight actions and events related to

FAA’s ODA program. We are also undertaking additional analyses of FAA’s

processes for certifying the 737 MAX 8 aircraft, including its use of the ODA

program, as well as examining FAA’s actions following the Ethiopian Air crash. We

will report on the results of these and other related reviews in future reports.

We conducted this audit in accordance with generally accepted Government

auditing standards. Exhibit A details our scope and methodology, and exhibit B

lists the organizations we visited or contacted. For a glossary of terms used in this

report, see exhibit C.

We appreciate the courtesies and cooperation of Department of Transportation

(DOT) representatives during this audit. If you have any questions concerning this

report, please call me at (202) 366-1959.

cc: The Secretary

DOT Audit Liaison, M-1

FAA Audit Liaison, AAE-100

AV2020037 3

Background

FAA is charged with overseeing the safety and certification of all civilian aircraft

manufactured and operated in the United States.

This is a significant

undertaking given that the U.S. civil aviation industry encompasses almost

292,000 aircraft, nearly 1,600 approved manufacturers, and more than

5,400 aircraft operators, among others. Recognizing that it is not possible for FAA

employees to oversee every facet of such a large industry, Federal law

allows the

Agency to delegate certain functions to private individuals or organizations, such

as determining compliance with aircraft certification regulations. Designees can

perform a substantial amount of critical certification work on FAA’s behalf. For

example, according to FAA data, in 2018 four U.S. aircraft manufacturers

approved about 94 percent of the certification activities for their own aircraft.

In 2009, FAA fully implemented the ODA program to standardize its oversight of

organizations (e.g., aircraft manufacturers) that are approved to perform certain

delegated functions on its behalf. While delegation is an essential part of meeting

FAA’s certification goals, the Agency faces challenges in providing ODA oversight.

For example, in 2015 we reported

that FAA’s oversight of ODA program controls

was not systems- and risk-based,

as recommended by an aviation rulemaking

committee.

Rather, the oversight was more focused on individual engineering

projects and areas that we determined were low risk. Under FAA’s ODA program,

the Agency’s Boeing Aviation Safety Oversight Office (BASOO) provides oversight

of authorized functions granted to Boeing. The BASOO is comprised of 42 FAA

employees who oversee Boeing’s ODA.

The Boeing ODA unit includes

approximately 1,500 Boeing-designated ODA representatives. FAA’s oversight

program is based on managing and supervising an organization, rather than

overseeing individual designees.

When undertaking certification activities for a manufacturer with an ODA, FAA

typically retains some level of involvement in significant design changes, novel

49 U.S.C. § 44702.

49 U.S.C. § 44702(d).

FAA Lacks an Effective Staffing Model and Risk-Based Oversight Process for Organization Designation Authorization

(OIG Report No. AV2016001), October 15, 2015. OIG reports are available on our website at http://www.oig.dot.gov/

Systems-based oversight shifts from focusing on individual project engineering work to holistically assessing

whether ODA companies have the people, processes, procedures, and facilities in place to produce safe products, thus

allowing FAA to focus its oversight on the highest-risk areas, such as new, innovative aircraft designs.

Aircraft Certification Process Review and Reform Aviation Rulemaking Committee, a joint FAA and industry group,

formed in response to a congressional mandate to study the aircraft certification process.

The BASOO includes 23 engineers who perform both certification work as well as oversight, 3 inspectors that

perform oversight, and additional project manager engineers and support staff.

AV2020037 4

designs, and critical compliance activities, based on the ODA’s experience and

FAA’s judgment of the potential risk. According to FAA, the Agency always retains

inherently governmental functions such as regulatory exemptions and functions

for which an ODA is not authorized.

FAA’s process for determining the certification basis of aircraft models is set forth

in regulations (14 CFR Part 21) and guidance (FAA Orders 8110.48 and 8110.4c).

Under this guidance, FAA can either award a type certificate

(TC) for new aircraft

models or an amended type certificate (ATC) for aircraft models that are

derivatives of already-certificated aircraft

(see figure 1 for a flowchart of the

certification process).

Figure 1. Key Phases in the Certification Process

Source: OIG analysis of FAA Order 8110.4C

An approval document issued by FAA that states a specific aircraft model is compliant with airworthiness

regulations.

This is known as the “baseline aircraft.”

AV2020037 5

The Boeing 737 MAX series

is the fourth-generation model of Boeing’s 737

aircraft series (see figure 2). The first Boeing 737, the 737-100, received its type

certificate on December 15, 1967—49 years before the Boeing 737 MAX 8. The

737 MAX 8 was certified as an ATC with the 737-800 (certified on March 13, 1998)

as the baseline, part of the 737 Next Generation (NG) series.

Figure 2. Boeing 737 Family of Aircraft – 1967 to 2017

Source: OIG analysis of FAA and Boeing data

According to FAA regulations, once applicants file for a new or amended type

certificate, they have 5 years to complete the process. During the certification

process, manufacturers are required to demonstrate compliance to the relevant

standards. Those standards are largely contained in 14 CFR Part 25 and are

amended as needed due to new technologies, in response to operational data, or

because of legislative mandates. The major milestones and requirements of the

certification process for a new or amended type certificate are similar. However, if

an aircraft model is certified under the ATC process, only systems or areas that

have been significantly changed need to be brought up to current regulatory

standards,

and other exceptions can be applied.

The 737 MAX 8 included a function in the flight control software—the

Maneuvering Characteristics Augmentation System (MCAS)—that was new to

commercial aircraft. MCAS modifies aircraft handling characteristics as an

additional function of the existing aircraft speed trim system. The speed trim

system is a flight control system designed to improve the airplane’s flight stability

The 737 MAX series includes the 7, 8, 9, 10, and 8200. The MAX 7, 10, and 8200 have not yet been certified by FAA.

14 CFR § 21.101 and Advisory Circular 21.101-A, more commonly known as the “Changed Product Rule.”

Applicants can also comply with earlier requirements when (1) an area, system, component, equipment, or

appliance are not affected by the change; (2) compliance with a later amendment does not materially improve safety;

or (3) compliance with the latest amendment is impractical.

AV2020037 6

during operations in certain conditions when the autopilot is not engaged.

oeing developed MCAS for the 737 MAX 8

to compensate for changes in

aerodynamics from the previous model caused by the MAX’s larger engines and

the placement of those engines on the wing (see figure 3).

Figure 3. Engine Size and Placement: 737 NG (pictured left) vs. 737

MAX (pictured right)

Source: Boeing

More specifically, MCAS can cause the airplane’s horizontal stabilizer

to move

without pilot input in certain, limited aircraft configurations

related to airspeed

and the angle of the aircraft in the air. This has the effect of moving the plane’s

nose down during flight (see figure 4) to compensate for the aircraft’s tendency

to pitch up. The accident report for the October 29, 2018, Lion Air accident states

that MCAS was a significant contributing factor for the accident, after activating

24 times during the flight.

MCAS activated after receiving faulty data from one

of the aircraft’s two Angle-of-Attack (AOA) sensors—external sensors that

measure the angle of the aircraft in the air. While the accident investigation for

the March 10, 2019, Ethiopian Air accident is still ongoing, the preliminary and

interim reports

also point to MCAS as a potential contributing factor to the

accident.

The speed trim system monitors airspeed, thrust lever position, and vertical speed, and then provides inputs to

adjust the aircraft’s horizontal stabilizer as needed. As the airplane speed increases or decreases, the system

automatically commands the stabilizer in the direction needed.

While MCAS is included on some military versions of the 767 refueling tanker, the system has different features on

that model, including additional redundancy of input data.

A control surface near the tail of the airplane that controls up and down movement of the airplane.

These configurations include the plane being in manual flight (autopilot off) and the flaps being in an up position.

Komite Nasional Keselamatan Transpotasi Republic of Indonesia. KNKT.18.10.35.04. FINAL. 2019.

Federal Democratic Republic of Ethiopia, Ministry of Transport, Aircraft Accident Investigation Bureau. Preliminary

Report. Interim Report, March 2020.

AV2020037 7

Figure 4. How MCAS Works on the 737 MAX

Source: OIG analysis of FAA and Boeing data

AV2020037 8

Results in Brief

Summary of Events Leading up to FAA’s Certification of the Boeing 737

MAX 8

Overall, FAA followed its established certification process for the 737 MAX 8,

which began in early 2012 when Boeing submitted its initial application for an

ATC. Under an ATC, as agreed to by FAA and Boeing, only the significant

differences between the 737 MAX 8 and the previous model—in this case, the

737-800 aircraft—must be certified to current regulatory standards as of the

application date. Early in the process, Boeing included limited information in

initial briefings to FAA on the MAX’s flight control software, MCAS, which

subsequently has been cited as a contributing or potentially contributing factor in

both accidents. However, Boeing presented the software as a modification to the

existing speed trim system that would only activate under certain limited

conditions. As such, MCAS was not an area of emphasis in FAA’s certification

efforts and therefore did not receive a more detailed review or discussion

between FAA engineers and Boeing. Instead, FAA focused its efforts on areas it

identified as potentially high risk, such as the aircraft’s larger engines, fly-by-wire

spoilers, and landing gear changes. As a result, FAA was not well positioned to

mitigate any risks related to MCAS.

From 2012 to 2014, Boeing and FAA collaborated to develop and implement an

overall certification plan, including determining which aspects of the certification

process would be delegated to the Boeing ODA. Throughout 2015 and 2016, FAA

and the Boeing ODA conducted certification activities that evolved over the

course of the project. During this timeframe, Boeing also began modifying MCAS

as a result of flight testing, including significantly increasing MCAS’s ability to

lower the aircraft’s nose automatically under certain conditions. However, Boeing

did not submit certification documents to FAA detailing the change. FAA flight

test personnel were aware of this change, but key FAA certification engineers and

personnel responsible for approving the level of airline pilot training told us they

were unaware of the revision to MCAS. Boeing did not communicate to FAA the

formal safety risk assessments related to MCAS until November 2016 and January

2017, more than 4 years into the 5-year certification process. According to FAA

management, it is not unusual for manufacturers to complete and submit the

safety assessments towards the end of the certification process. Moreover,

Boeing’s safety analysis did not assess system-level safety risks as catastrophic;

thus, Boeing designed MCAS to rely on data from a single aircraft sensor rather

than including redundancy, which would have reduced risk.

In 2016, FAA and Boeing began certification flight testing to determine the

aircraft’s compliance with FAA’s requirements. In addition, FAA’s Flight Standards

Service conducted separate tests and subsequently approved a training plan

AV2020037 9

proposed by Boeing—known as Level B training—for 737 MAX pilots who were

already qualified to fly the Boeing 737-800. This outcome aligned with Boeing’s

overarching goal of achieving a common type rating

for pilots moving from the

NG series to the MAX and keeping costs down by avoiding simulator training for

MAX pilots. Pilot response to automated MCAS activation was not included in the

required training. In March 2017, FAA issued an ATC to Boeing for the 737 MAX

8, which began flying passengers later that year.

Summary of Events Between the Lion Air and Ethiopian Airlines Crashes

On October 29, 2018, Lion Air Flight 610 crashed, resulting in 189 fatalities.

According to the accident report, MCAS activated based on erroneous AOA data

more than 20 times, automatically pushing down the aircraft’s nose, before the

flight crew lost control. According to Boeing, while its engineers and test pilots

had anticipated multiple MCAS activations to be possible, they decided it would

be no worse than a single activation because pilots would be able to recognize

and counteract any downward movement of the aircraft’s nose. On November 6,

2018, Boeing generated a bulletin

to operators, prompting FAA to issue an

Emergency Airworthiness Directive (AD) the next day. Although the bulletin and

the Emergency AD emphasized pilot procedures for handling repeated nose-

down movements, neither specifically mentioned MCAS. At this time FAA also

began reviewing the MCAS certification process. This was the first time that FAA’s

certification engineers had performed a detailed review of MCAS, and according

to several FAA certification engineers, it was also the first time they were

presented with a full picture of how MCAS worked. As a result of FAA’s risk

analysis following the crash, Boeing proposed, and FAA accepted, a redesign of

MCAS. In February 2019, FAA and Boeing formally agreed on a schedule to

implement the recommended MCAS software fix. Based on FAA’s risk analysis

and existing risk guidelines, Boeing was to complete the software update by April

12, 2019. On March 10, 2019, Ethiopian Air Flight 302, operating a 737 MAX 8

aircraft, crashed shortly after departing Addis Ababa Bole International Airport,

resulting in 157 fatalities, including 8 Americans. The interim accident report

shows that MCAS activated based on erroneous AOA data before the crew lost

control.

Summary of Concurrent FAA ODA Oversight Actions and Events

During the same time period as FAA’s certification efforts, Boeing, FAA, and our

office were identifying issues that—although not specific to the 737 MAX 8—may

have impacted the original certification of the aircraft. More specifically, in 2015,

we reported on FAA’s lack of a risk-based oversight approach to ODA. In

A type rating is an endorsement on the pilot certificate indicating that the pilot has completed the required training

and testing for a specific make, type, and/or series of aircraft (for example Boeing 747-400).

Flight Crew Operations Manual (FCOM) Bulletin TBC-19, dated November 6, 2018.

AV2020037 10

addition, FAA identified concerns regarding the quality of ODA certification

documents that needed to be addressed. In December 2015, FAA and Boeing

signed a Settlement Agreement, and Boeing paid a civil penalty of $12 million

regarding violations of Boeing’s quality control system and insufficient

certification documents. FAA’s subsequent oversight found that Boeing has not

yet resolved all the identified issues, including improving its identification and

resolution of the root causes of non-compliances with FAA requirements. During

this time period, Boeing and FAA also identified concerns about undue pressure

on ODA personnel at multiple Boeing facilities, which culminated in FAA issuing a

formal compliance action against Boeing in November 2018. Boeing’s response

to this compliance action remains ongoing.

Given that we have an open recommendation for FAA related to ODA and that

we are planning additional analysis of FAA’s certification process and the use of

the ODA program for the 737 MAX 8 aircraft, we are not making

recommendations in this report. The data gathered are informational and

represent our observations in response to the Secretary’s and other congressional

requests. We will report further on FAA’s oversight of the certification process

and other related matters, as well as make recommendations as applicable, in

future reports.

Timeline of Activities Leading to the Certification of

the Boeing 737 MAX 8 Aircraft

The following presents a detailed timeline of the events leading to the

certification of the 737 MAX 8, which began in January 2012 and culminated with

the issuance of an ATC in March 2017. Figure 5 presents an overview of the

events; detailed descriptions follow.

AV2020037 11

Figure 5. Timeline of Major Events for the Certification of the Boeing 737 MAX 8

Source: OIG analysis of FAA and Boeing data

AV2020037 12

2012–2013: Boeing and FAA Initiate Certification

Process; MCAS Was Not an Area of FAA Emphasis

The certification process for the 737 MAX 8 officially began in early 2012, when

Boeing submitted its initial application for an ATC. In November 2013, FAA

accepted Boeing’s 737 MAX 8 Master Certification Plan, which established the

means of compliance Boeing planned to use to get the aircraft design certified.

While Boeing’s flight control software, MCAS, was included in an early technical

briefing presented to FAA, MCAS was not an area of emphasis because Boeing

presented it to FAA as a modification to the existing speed trim system, with

limited range and use. According to FAA, the Agency focused its involvement on

potentially higher risk areas such as the aircraft’s larger engines, fly-by-wire

spoilers, and landing gear changes.

January 27, 2012

Boeing files an Amended Type Certificate application with FAA for the 737

MAX 8.

In submitting its application for certification, Boeing used the 737-800 aircraft

design as the basis for the 737 MAX 8 model. Under FAA’s aircraft certification

processes,

FAA can prescribe a special condition for new or novel technology

when no applicable standards exist. Systems identified as new or novel receive

extra scrutiny from FAA. According to Boeing, the company did not need to

identify MCAS as new or novel and MCAS did not require a special condition. The

company stated this was because the design feature had been covered under

existing regulations

relating to flight control systems, in addition to being

included on the military Boeing 767 refueling tanker. However, the version of

MCAS installed on the 767 tanker differed from the version of MCAS installed on

the 737 MAX. For example, while both aircraft have two AOA sensors, the tanker

version uses a median input value of both sensors, while the 737 MAX version of

MCAS relied on the input from one sensor.

Moreover, the Joint Authorities

14 CFR § 21.16 states that if applicable regulations do not contain adequate or appropriate safety standards for an

aircraft due to a novel or unusual design feature, FAA can prescribe a special condition to the aircraft to ensure an

equivalent level of safety to requirements in the regulations.

Existing regulations including 14 CFR §§ 25.671, 25.672, 25.1309, and 25.1329.

The MCAS software resides in both of the aircraft’s flight control computers, each of which receives data input from

its respective AOA sensor. The 737 MAX is designed to rely on only one of its two flight control computers per flight,

alternating from one to the other after each flight. As a result, MCAS receives data from just one AOA sensor.

AV2020037 13

Technical Review (JATR)

team stated in its 2019 report that MCAS on the 737

MAX controlled the aircraft’s movements in a new way.

ccording to internal Boeing meeting minutes from 2013,

the company made

the decision to portray MCAS as a modification to an existing flight control

system in part because if MCAS “was emphasized as a new function, there may

be a greater certification and training impact.” An ODA representative working on

FAA’s behalf also agreed with portraying MCAS as a modification and not a new

function. According to an FAA Flight Standards representative and an internal

Boeing email, an early Boeing program goal was to keep a common type rating

for the aircraft—which would minimize additional training requirements for 737

MAX pilots previously certified on the NG series—and to avoid the need for 737

MAX pilots to train in simulators, which can add costs for airlines that purchase

the aircraft. References to MCAS were later removed from flight crew training

requirements; therefore, any simulator training, while not proposed, probably

would not have included MCAS.

March 21, 2012

FAA and Boeing hold a General Familiarization Meeting.

During General Familiarization Meetings, the certificate applicant introduces FAA

to the changes or new systems and features of an aircraft design and reviews the

general architecture. Under an ATC, only the significant differences from the

baseline model must be certified to the regulations applicable on the date of the

application. (See figure 6 for a diagram of what FAA identified as the significant

changes to the aircraft. MCAS was included in “system revisions.”) One FAA flight

control engineer we interviewed recalled that during the 737 MAX General

Familiarization meeting he participated in, MCAS information was not an area of

focus, but it was presented briefly with limited details. Technical familiarization

documents we reviewed supported this evidence.

The JATR is a team consisting of representatives of regulators from 10 countries (including the United States) that

was chartered by FAA on June 1, 2019, to examine the Agency’s certification of the 737 MAX 8. The JATR issued a

report on October 11, 2019.

This particular meeting was held June 7, 2013, but represents an example of how Boeing presented the system to

FAA and other regulators in order to meet program goals.

AV2020037 14

Figure 6. Significant Changes From the 737 NG to the 737 MAX Aircraft

Source: OIG analysis of FAA and Boeing data

March 22, 2012

FAA accepts Boeing’s Amended Type Certificate application.

One day after FAA and Boeing had their General Familiarization Meeting, FAA

acknowledged Boeing’s proposal for the 737 MAX 8 and allowed the applicant to

proceed. MCAS was included in the accepted ATC application as a modification

to the previous aircraft model’s flight control system software.

May 1–2, 2012

FAA and Boeing hold a Technical Familiarization Meeting.

Similar to General Familiarization Meetings, Technical Familiarization Meetings

are opportunities for FAA to learn about technical designs and changes from the

applicant. During the meetings, FAA typically looks at the design changes

between the new derivative aircraft and the baseline aircraft and determines what

issues must be addressed. In addition, FAA began evaluating whether to delegate

or retain authority for assessing whether specific areas, features, or systems

comply with Federal regulations.

In Boeing’s Technical Familiarization Meeting presentations, MCAS was included

as a provisional modification to address the plane’s tendency to pitch upwards at

high speeds. However, according to FAA representatives present at the meeting,

it was not an area of emphasis. Based on our review of the Technical

AV2020037 15

Familiarization Meeting briefing slides, we determined that 23 of 482 slides

covered primary aircraft flight controls. However, there were only 2 lines of text

within those almost 500 slides—covered over a 2-day period— that referenced

MCAS. According to FAA representatives, flight control engineers on the project

focused on other issues identified as potentially higher risk, such as fly-by-wire

spoilers.

This demonstrates that FAA relied heavily on the information that

Boeing provided in the early stages of the aircraft’s certification, which in turn

drove the Agency’s decisions on areas of involvement.

uly 21, 2012

FAA begins the initial certification basis evaluation.

This evaluation is FAA’s initial review of Boeing’s proposed certification basis—

i.e., how the company proposed to satisfy the certification requirements for its

737 MAX 8. Within the certification basis document, FAA requires that applicants

provide an adequate overview of the project scope to support early certification

activities.

February 15–November 14, 2013

FAA reviews and accepts the Master Certification Plan.

The Master Certification Plan is a key step in the certification process, as this

document describes how FAA and Boeing planned to certify the 737 MAX 8,

including the method for testing key items. This document establishes which

project areas and documents Boeing proposes its ODA will be responsible for

reviewing and assessing for compliance (i.e., which items FAA will delegate to the

ODA) and which areas and documents FAA will retain or remain involved in.

According to our review, FAA and Boeing communicated questions, changes, and

clarifications with each other as Boeing worked to develop the Master

Certification Plan. In November 2013, the Agency accepted the Master

Certification Plan. FAA initially retained the sections of the Master Certification

Plan related to flight controls and the stabilizer, including MCAS. At this time,

according to Boeing data, FAA had delegated 28 out of 87 (32 percent) detailed

certification plans for the aircraft.

Notably, the number of certification plans that FAA delegated and retained

changed throughout the certification process, which, according to FAA, is typical.

For example, according to Boeing data, as of November 2016 FAA had delegated

79 of 91 (87 percent) detailed certification plans back to Boeing’s ODA, including

the flight controls and stabilizer plans containing MCAS (see figure 7).

In a fly-by-wire system, a computer collects sensor data from the pilot’s controls and uses those signals to move

the corresponding aircraft control surfaces; this replaces an older system relying on physical cables connecting pilot

controls to control surfaces.

According to Boeing data, between November 2016 and March 2017, FAA eventually delegated all 91 certification

plans to Boeing’s ODA.

AV2020037 16

Furthermore, under FAA’s ODA program, FAA can delegate specific deliverables

within each certification plan, such as system safety assessments, even if FAA

retains the plan itself. These can also change over the course of the project, as

was the case for the over 1,700 Boeing 737 MAX deliverables.

Figure 7. Delegation and Retention of Certification Plans

32%

68%

Certification Plans at Master

Certification (November 2013)

Certification Plans Delegated

Certification Plans Retained

87%

13%

Certification Plans as of

November 2016

Certification Plans Delegated

Certification Plans Retained

Source: OIG analysis of Boeing data

2014: FAA Establishes Initial Certification Basis

February 6, 2014

FAA establishes the initial certification basis for the 737 MAX 8.

Over approximately 2 years between 2012 and 2014, Boeing and FAA

collaborated to establish the initial certification basis for the 737 MAX 8, having

agreed in March 2012 that an ATC would be appropriate for the aircraft. During

this same time period, FAA and Boeing collaborated through Issue Papers, which

provide a continuous way of communicating and working through differences

about the means of compliance with relevant standards and regulations.

FAA

established the certification basis for the model 737 MAX 8 within the G-1 Issue

Paper, a document that FAA and Boeing used to collaborate about design

requirements and conditions. The initial certification basis specifies the applicable

regulations and special conditions that must be complied with for the project.

Between this date and March 2, 2017, FAA and Boeing continued formally

In Order 8110.112A, FAA defines Issue Papers as the method to document the negotiation and resolution of

certification issues with the applicant.

AV2020037 17

discussing the certification process through the G-1 Issue Paper, refining the

certification basis accordingly.

2015: Certification Continues, Including Revisions

to MCAS

Throughout 2015, FAA and Boeing’s ODA unit members

continued certification

activities for the 737 MAX 8. Boeing also began revising MCAS based on the

results of aircraft analyses and testing. However, MCAS was still not a major focus

of FAA’s certification efforts, which continued to emphasize areas such as the

aircraft’s new larger engines, fly-by-wire spoilers, and changes to the landing

gear.

September 17–18, 2015

Joint Operational Evaluation Board reviews Boeing’s Flight Crew Operations

Manual (FCOM), including MCAS.

The FCOM is an aircraft-specific manual that manufacturers provide to operators,

which contains necessary operating limitations and other procedures the flight

crew needs to safely operate that aircraft. As part of the certification process, a

panel of representatives from FAA, Transport Canada, and the European Aviation

Safety Agency, known as the Joint Operational Evaluation Board, reviewed the

draft FCOM on September 17–18, 2015. The draft FCOM at the time included a

brief description of MCAS indicating that it would only activate at high speed and

high load factors.

It did not include any references to repeated activations.

According to Boeing, MCAS was removed from the FCOM in March 2016.

Subsequent versions that we reviewed did not include a description of MCAS but

retained MCAS in the abbreviations section.

2016: Boeing and FAA Conduct Flight Tests;

Boeing Completes Failure Analysis and Continues

To Revise MCAS

Flight testing of the 737 MAX 8 began in 2016. Boeing also completed failure

analyses during the same time period. Flight testing is a critical component of the

certification process. Typically, flight testing involves a series of tests in order to

verify engineering assumptions, assess design decisions, and check for

ODA unit members are organization employees authorized, as part of the ODA and on FAA’s behalf, to perform

functions necessary for FAA approval of that project as outlined in FAA Order 8100.15B.

Load factor is the ratio of aerodynamic forces divided by the weight of the aircraft.

AV2020037 18

compliance with requirements related to flight handling, stall identification, and

control system malfunctions.

January 19, 2016

Boeing completes the first Single and Multiple Failure document on the 737

MAX 8.

Boeing uses the Single and Multiple Failure document to “prevent simultaneous

failure from a single threat event which causes loss of continued safe flight and

landing.” Boeing considered this failure probability analysis an internal document

only and did not submit it as a required certification deliverable. Therefore,

Boeing did not provide it to FAA, nor did FAA have to review or approve it as part

of the certification process. However, according to FAA, some aspects of Boeing’s

analysis from the Single and Multiple Failure document should be included in

system safety assessments later provided to the Agency as certification

deliverables.

Boeing’s analysis identified 75 failure cases to assess the potential impacts of

those failure scenarios on the aircraft and flight crew. Boeing deems failures to be

acceptable under certain circumstances, including if:

• the equipment and systems perform as intended during operating

conditions;

• catastrophic failure conditions are not caused by a single failure;

• catastrophic failures are determined to be extremely improbable;

• information concerning unsafe system operating conditions is provided to

the crew.

Boeing’s Single and Multiple Failure analysis found all 75 potential failure cases to

be acceptable.

One potential failure case involved the loss of one AOA sensor—an external

sensor that measures the angle of the aircraft in the air

—followed by faulty

AOA data in the other sensor. (See figure 8, which shows the location of AOA on

the 737 MAX aircraft.)

AOA sensors are attached to the outside of the aircraft.

AV2020037 19

Figure 8. 737 MAX Aircraft External Probes, Including the AOA

Sensor

The AOA sensor is the lower one of the two probes.

Source: Boeing

However, despite identifying this failure case and deeming it catastrophic, Boeing

determined this failure case was acceptable because the probability of occurrence

was determined to be extremely remote, and it was assumed the crew would

recognize the situation and take appropriate action. While this failure test case

may not be exactly the same as the circumstances encountered in the Lion and

Ethiopian Air accidents, erroneous AOA data—potentially caused by the failure of

one AOA sensor—was a factor present in both accident scenarios.

During its Single and Multiple Failure analysis, Boeing rated this potential failure

case as “catastrophic”

but also determined that the low probability of

occurrence meant it qualified as acceptable. Boeing also rated 11 other potential

failure cases as catastrophic but ultimately deemed them acceptable based on

probability and engineering judgement.

FAA Advisory Circular 25.1309.1A classifies risk ratings as: Minor (failure conditions which would not significantly

reduce airplane safety and which involve crew actions that are well within their capabilities), Major (failure conditions

which would reduce the capability of the airplane or the ability of the crew to cope with adverse conditions),

Hazardous (failure conditions which would reduce the capability of the airplane or the ability of the crew to cope due

to physical distress or excessive workload), and Catastrophic (failure conditions which would prevent continued safe

flight and landing).

AV2020037 20

January 29, 2016

Boeing conducts first flight test of the Boeing 737 MAX 8.

Boeing’s 737 MAX 8 flight testing began in January 2016. However, per FAA’s

standard certification process, FAA does not actively participate in flight testing

until the Agency issues the Type Inspection Authorization (TIA).

March 14, 2016

FAA issues initial Type Inspection Authorization.

The initial TIA authorized Boeing to begin certification test flights for the 737

MAX 8 using FAA flight test pilots, and defined which certification test flights FAA

needed to perform. The flight test protocol included 129 certification flight test

plans to be completed as part of the testing. Under this TIA, FAA delegated 62 of

the 129 (48 percent) flight test plans to Boeing’s ODA.

The TIA also defined the tests and analysis that would be performed by the

Seattle Aircraft Evaluation Group (AEG)—a group of aviation safety inspectors in

FAA’s Flight Standards Service tasked with determining the appropriate type

rating and levels of training for aircraft that are undergoing evaluation for an

ATC. For the 737 MAX 8, as required, the AEG assessed and determined it needed

to participate in its own testing. The testing that the AEG performed was handling

and differences testing, which determines the type rating and the differences

between the baseline aircraft (737-800) and related aircraft (737 MAX 8) on which

pilots must be trained, evaluated, and remain current.

March 30, 2016

Boeing completes MCAS Revision D, a significant change.

Boeing continued to revise and refine MCAS during the flight testing process.

Revision D was a significant change that updated MCAS software technical and

data requirements based on pilot assessments and flight test results. In this

revision, Boeing changed the parameters under which MCAS would activate to

include much slower airspeeds.

It also increased the maximum range of MCAS

from 0.55 degrees to 2.5 degrees, an increase of over 300 percent. This meant

that each time MCAS activated, it could push the nose of the aircraft downward

with a maximum range of 2.5 degrees of movement.

In its MCAS Revision D, Boeing also included an assessment of functional hazards

related to the software, describing hazard descriptions, failure conditions, and

Guidance for flight testing of the 737 MAX 8 is contained in FAA Advisory Circular 25-7C.

Following this revision, MCAS could now activate at speeds of 0.2 to 0.84 Mach, whereas it could previously only

activate at speeds above 0.67 Mach. Mach is calculated by dividing the speed of the aircraft by the speed of sound.

Specifically, MCAS uses the aircraft’s horizontal stabilizer (near the tail of the aircraft) to control the angle of the

aircraft, and the range of MCAS is the angle of movement of the stabilizer.

AV2020037 21

associated effects. One of the noted hazards was an uncommanded or automatic

MCAS activation that continued until the pilot took action. When developing this

risk assessment, Boeing tested unintended MCAS activation in the simulator and

assumed that commercial pilots would recognize the effect as a runaway

stabilizer

—a scenario which is covered in basic commercial pilot training—and

react accordingly. Boeing assumed the average pilot reaction time in this scenario

to be 4 seconds, which Boeing classified as a hazardous

event. However, if a

pilot’s reaction time was greater than 10 seconds, the event would be classified

as catastrophic due to the pilot’s inability to regain control of the aircraft. Despite

these significant revisions, Boeing did not provide internal coordination

documents for Revision D,

noting the increased MCAS range, to FAA

certification engineers. Because these revision documents were not required

certification deliverables, the company did not submit them to FAA for review or

acceptance.

March 30, 2016

Boeing removes MCAS from the Flight Crew Operations Manual and training

differences tables.

Boeing requested (and received) permission from the FAA AEG to remove any

references to MCAS from its FCOM

and Other Differences Requirements tables,

which help aircraft operators manage their training on differences between

related aircraft. Boeing justified its request by stating that MCAS would be

“transparent”

to the flight crew. This decision was not documented via official

correspondence; rather, Boeing requested it via email and FAA approved it

verbally in a subsequent meeting, according to the responsible FAA AEG

representative.

However, the FAA AEG representative who concurred with the request told us

that FAA based the decision on the understanding that MCAS was still as

originally designed—i.e., that it would only activate in situations that included

high speeds and high load factors, and was limited to a range of 0.55 degrees. An

earlier version of the FCOM that this representative would have reviewed

contained only three sentences specifically referencing MCAS, and it stated that

the system would only activate when the aircraft was at speeds of “0.7 Mach or

A technical fault resulting in continuous unintended movement of the horizontal stabilizer.

Boeing added a statistical credit in its evaluation of this scenario that reduced the effect from Hazardous to Major,

based on the assumption that it was unlikely that a typical flight would be operating outside of normal aircraft

parameters.

Revision D is where the major changes to MCAS first occurred; subsequent MCAS Revision E (dated July 5, 2016)

configurations were the versions actually installed on the aircraft as of the date of ATC issuance.

FAA does not formally approve the FCOM. However, Agency inspectors do review and accept the document before

it is issued to individual operators.

In this context, Boeing and FAA use the word “transparent” to mean the system or function would be invisible to

the flight crew—they would not be aware or have any indication that the system was present or in operation.

AV2020037 22

greater.” However, on this same date, Boeing revised MCAS to activate at much

slower speeds.

While AEG and FAA certification engineers were unaware of the revisions to

MCAS, FAA flight test personnel were aware of the increased maximum range of

MCAS in the flight control computer actually installed on the 737 MAX 8 test

aircraft. This varied understanding of the final flight control design of the 737

MAX among different FAA offices demonstrates a lack of consistent and

transparent communication both between Boeing and FAA, as well as within FAA.

April 13, 2016

FAA conducts first flight test of the 737 MAX 8.

Following the issuance of the TIA, FAA conducted flight tests using the Agency’s

test pilots from the Aircraft Certification Service. The flight crews for tests typically

included one FAA test pilot, one FAA test flight engineer, Boeing engineers, and

one Boeing pilot.

May 6, 2016

Flight Control System (including MCAS) flight testing begins.

FAA began a series of seven flight tests to test stall speed performance for the

737 MAX 8. While MCAS was not tested as a stand-alone item, FAA stated that it

included MCAS in its tests of the flight control system and stall functions. For

example, according to FAA, the Agency tested several high-speed maneuvers—

such as upset recovery

and wind-up turns

—in which MCAS would have

activated as intended, but was not the focus of the test.

August 15, 2016

Boeing releases Flight Control Computer software revision.

Boeing released the version of the flight control computer software that it

intended to use as the final version on the MAX 8 on this date. This version is

known as the “Black Label Equivalent,” meaning the software has undergone

additional testing but requires a final flight test conducted by FAA. This software

revision included the version of MCAS that had the ability to push down the nose

of the aircraft with a maximum movement of 2.5 degrees.

Upset recovery is the ability to correct the aircraft after unintentionally exceeding normal flight parameters.

A wind-up turn is a constant altitude, constant speed turn with increasing normal acceleration or angle of attack.

AV2020037 23

August 16, 2016

Boeing completes requirements for aircraft Level B training.

Boeing successfully completed the testing requirements for Level B training on

August 16, 2016. FAA’s Boeing 737 Flight Standardization Board (FSB)

approved

Level B training

for the 737 MAX 8 for pilots who were qualified to fly the 737-

800.

This action meant that pilots would be eligible to fly the 737 MAX 8

following completion of classroom or computer-based training on FAA-

mandated topics. Level B training also meant that no simulator training was

required (see table 1 for pilot training differences levels). FAA granted this

approval after using domestic, commercial pilots—in addition to FAA operational

test pilots—during its training-related flight testing, a practice that is allowed but

not frequently done.

The approved training did not include material on MCAS. Internal Boeing emails

show company officials congratulating staff for the accomplishment of receiving

FAA approval for Level B training, providing further evidence that this reduced

level of training was a Boeing program goal for the 737 MAX.

FAA typically establishes an FSB when certificating large jet or propeller aircraft. It consists of members of the AEG,

FAA operations inspectors for the initial operator of the aircraft, representatives from the Office of Safety Standards,

and other technical advisors if necessary. One of the FSB’s mandates is to develop training objectives for normal and

emergency procedures and maneuvers.

FAA uses AC 120-53B to determine the level of training necessary for pilots that hold a type rating in a particular

aircraft to be able to obtain a type rating in a related derivative aircraft.

FAA issued a provisional training validation letter on February 22, 2017, and formally approved the training on

March 7, 2017.

AV2020037 24

Table 1. Pilot Training Difference Levels

Level Training Training Methods Checking

A Self-instruction Operating manual page revisions, operating bulletins,

handouts, etc.

Not applicable (or next

proficiency check)

B Aided instruction

Slide/tape presentations, computer-based instruction,

stand-up lecturers, or video tapes, etc.

(Note: This was the level of training approved for the

Boeing 737 MAX.)

Task or system check

C Systems devices Training devices to supplement instruction, including

interactive computer-based training, cockpit procedure

trainers, or part task trainers.

Partial proficiency check

using device

D Partial flight

simulator

Flight training device that is accurate, capable of

performing flight maneuvers in a dynamic real time

environment, high fidelity integration of systems and

controls, and realistic instrument indications.

Partial proficiency check

using Flight Simulation

Training Device

Full flight simulator

or aircraft

Requires full flight simulator or aircraft training. New

type rating is normally assigned.

Proficiency check on full

flight simulator, or

aircraft

Source: OIG analysis of FAA policy

September 28, 2016

FAA delegates Flight Control Computer Certification Plan, which includes

MCAS, to Boeing’s ODA.

Initially, the Master Certification Plan stated that FAA engineers would be

responsible for reviewing a key safety assessment of the flight control system,

including MCAS. According to FAA representatives, however, Boeing did not

present the formal version of this system safety assessment to FAA for the first

time until January 2017—more than 4 years into the 5-year certification process.

According to FAA management, it is typical for manufacturers to complete and

submit the safety assessments towards the end of the certification process. Prior

to this, on September 28, 2016, FAA engineers decided to delegate back approval

of future certification plan revisions to the Boeing ODA, but FAA continued to

retain the safety assessment deliverable until the Agency reviewed it in February

2017.

November 10, 2016

FAA delegates Stabilizer Certification Plan to Boeing’s ODA.

While initially retained, FAA delegated to Boeing’s ODA the Stabilizer Certification

Plan, which details how the manufacturer will demonstrate compliance for the

AV2020037 25

aircraft’s horizontal stabilizer—a control surface near the tail of the airplane that

controls up and down movement of the airplane. When MCAS activates, it adjusts

the angle of the horizontal stabilizer on the plane, resulting in pushing down the

nose of the aircraft. While Boeing’s Stabilizer Certification Plan document

included some details regarding MCAS functions, it did not include an

interrelated view of how MCAS interacted with other systems, which was spread

throughout several documents. FAA delegated the system safety assessment for

this certification plan back to Boeing the following month, in December 2016. The

FAA engineer who reviewed the system safety assessment stated that he was not

aware of the increased MCAS range from 0.55 degrees to 2.5 degrees on the

aircraft’s horizontal stabilizer when he recommended approval of Boeing’s

delegation request. In addition, Boeing did not update the hazard assessment

table within the safety assessment to reflect the expanded MCAS use and range.

In this system safety assessment, Boeing identified potential failure scenarios

related to the horizontal stabilizer and evaluated their risk. Notably, Boeing

included a scenario in which there would be an “unintended MCAS activation.”

However, Boeing assigned this failure scenario the risk rating of “Major” under

normal flight operations, which meant that there was no requirement to provide

design redundancy (i.e., a requirement for MCAS to pull data from both external

AOA sensors on the 737 MAX 8, rather than relying on a single AOA sensor as the

system was designed). Such redundancy is required for the higher-risk rating of

catastrophic.

Boeing recognized that the risk of unintended MCAS activation could be more

severe under certain circumstances if the aircraft was operating outside of normal

flight parameters. However, the company adjusted its evaluation of this risk

based on statistical analysis showing it was unlikely that a typical flight would be

operating in those circumstances, and therefore unlikely that MCAS would

activate under these conditions. In its 2019 report following the two accidents,

the JATR

questioned Boeing’s assumption, stating that the statistical credit

Boeing used was intended to be used in selecting test cases for flight handling

qualities evaluation, not for showing compliance with systems safety regulations.

While Boeing tested a single, unintended activation of MCAS, it did not test

repeated MCAS activations. Boeing engineers and test pilots, in discussions,

deemed multiple activations of MCAS to be no worse than a single activation of

MCAS. However, Boeing did not include this untested conclusion in certification

deliverables provided to FAA. Further, Boeing’s safety assessments did not fully

account for how pilots would react to a multi-failure scenario. Boeing noted in

Joint Authorities Technical Review (JATR), Boeing 737 MAX Flight Control System: Observations, Findings, and

Recommendations. Submitted to the Associate Administrator for Aviation Safety, U.S. Federal Aviation Administration

on October 11, 2019.

AV2020037 26

these assessments that it did not simulate an accumulation or combination of

failures leading to unintended MCAS activation, nor their combined flight deck

effects.

November 20, 2016

FAA and Boeing complete flight testing of Flight Control System (including

MCAS).

This was the final certification test flight for the control system that includes

MCAS. In sum, Boeing and FAA completed 58 tests of the airplane’s flight control

systems. FAA flight test pilots participated in 47 of the 58 flight control system

tests, and FAA flight test engineers participated in 34 of these tests. According to

FAA flight test personnel, some of the tests included MCAS operation with a

range of 0.55 degrees, while other tests were at 2.5 degrees. This varied

depending on which version of MCAS was installed on the aircraft used for the

test flight, as Boeing was revising MCAS during flight testing.

2017: FAA Certifies the Boeing 737 MAX 8

In March 2017, FAA issued an ATC to Boeing for the 737 MAX 8 aircraft, which

meant that the company could begin delivering the MAX 8 to customers. The

aircraft began flying commercially on May 22, 2017.

March 8, 2017

FAA issues the Amended Type Certificate, certifying the Boeing 737 MAX 8

meets transport category airplane requirements.

FAA completed the certification of the 737 MAX 8 and issued an ATC for the

aircraft.

Subsequently, FAA issued an ATC for the 737 MAX 9 on February 15,

2018. The other 737 MAX variants that have yet to be certified include the smaller

737 MAX 7, the larger 737 MAX 10, and the higher-seating capacity 737 MAX

8200.

May 16, 2017

Boeing delivers the first Boeing 737 MAX 8.

Boeing delivered the first Boeing 737 MAX 8 to Malindo Air, a subsidiary of Lion

Air Group. Malindo Air then flew the first Boeing 737 MAX 8 commercial flight on

May 22, 2017, from Kuala Lumpur to Singapore. Until the grounding order in

March 2019, Southwest, United, and American Airlines operated the 737 MAX 8

and 9 in the United States. Southwest received the first domestic delivery of a 737

Section 21.101e states if type certificate approval does not occur within 5 years, the applicant may select a new

application date, which revises the date of the applicable 14 CFR Part 25 regulations. The applicant then must comply

with any new or revised regulations as of the amended date. In December 2016, Boeing amended the application date

to June 30, 2012; thus, the certification was in compliance with the 5-year requirement.

AV2020037 27

MAX 8 on August 26, 2017. As of December 2019, a combined total of

387 Boeing 737 MAX 8 and MAX 9 aircraft had been delivered worldwide and

72 specifically for U.S. commercial air carriers (see tables 2 and 3).

Table 2. Boeing 737 MAX Aircraft, in Fleet and on Order, as of

December 2019

U.S. Commercial

Air Carriers

U.S. Non-Commercial

Operators

Foreign

Carriers Total

In Fleet 72 46 269 387

On Order 528 463 3,554 4,545

Source: OIG analysis of Boeing documents

Table 3. Boeing 737 MAX Aircraft, in Fleet and on Order for U.S.

Commercial Air Carriers, as of December 2019

Alaska American Southwest United Total

In Fleet 0 24 34 14 72

On Order 32 76 249 171 528

Source: OIG analysis of Boeing documents

August 10, 2017

Boeing identifies AOA disagree alert issue.

Boeing design engineers identified an issue with a cockpit alert designed to

notify pilots when the two AOA sensors disagree by more than 10 degrees for at

least 10 seconds. Disagreeing AOA sensors can indicate that one or more sensors

has failed or is providing unreliable information. However, Boeing discovered that

not all 737 MAX aircraft were equipped with the alert. According to Boeing

representatives, Boeing had intended this cockpit alert message to be standard

on all 737 MAX 8 aircraft within the flight control computer system. However,

Boeing stated that its software contractor inadvertently paired the alert with an

optional feature—an AOA indicator—which only approximately 20 percent of

MAX customers purchased

(see figure 9). Neither of the accident aircraft had

this AOA disagree cockpit alert.

Komite Nasional Keselamatan Transpotasi Republic of Indonesia. KNKT.18.10.35.04. FINAL. 2019.

AV2020037 28

Figure 9. AOA Disagree and AOA Indicator

Source: FAA

When first implemented on the 737 NG (and as carried over into the MAX), there

were no established Boeing operational procedures for the AOA cockpit alert

feature. Instead, it was a source of supplemental information, and the checklist

for the AOA disagree did not require any pilot action as a result of the alert.

Therefore, according to Boeing representatives, they analyzed the issue and

determined that the cockpit alert was not “necessary for the safe operation of the

airplane.” Boeing documented the problem in August 2017 and planned to have

the problem corrected for the entire MAX fleet by late 2020.

Boeing did not directly notify FAA of the AOA alert issue

since its analysis had

determined that there was not an “operational impact,” nor did the company

notify MAX operators at the time the issue was discovered. According to Boeing

representatives, the company will generally inform its customers of issues such as

this one. In this case, however, Boeing failed to do so upon discovering the

omission, and, according to Boeing representatives, the company has been

unable to determine why it did not notify operators about the AOA alert issue.

According to Boeing representatives, the AOA disagree cockpit alert message issue was included in updated

certification documents in October 2017; however, Boeing did not submit a formal notification of the issue directly to

FAA nor could the company confirm that FAA had reviewed the documents containing the first notification. According

to Boeing, the Agency agreed in February 2019 with Boeing’s disposition of the problem and the determination that it

was “not an unsafe condition.”

AV2020037 29

Boeing representatives stated that the company has since notified the operators

of the AOA disagree alert issue.

Timeline of Events Between the Lion Air and

Ethiopian Airlines Crashes

Following the fatal Lion Air crash on October 29, 2018—the first accident of a 737

MAX 8 aircraft—Boeing generated a bulletin to operators, prompting FAA to

issue an Emergency Airworthiness Directive (AD). FAA issues an Emergency AD

when an unsafe condition exists that requires immediate action by an

owner/operator. However, neither the bulletin nor the Emergency AD specifically

mentioned MCAS. Just over 4 months after the first accident, on March 10, 2019,

Ethiopian Airlines Flight 302 crashed, resulting in 157 fatalities.

October 29, 2018

Lion Air Flight 610 crashes, resulting in 189 fatalities.

Lion Air Flight 610 crashed into the Java Sea shortly after departing Soekarno-

Hatt International Airport, Jakarta, resulting in 189 fatalities. According to the

accident report, multiple cockpit alerts activated during takeoff, including a

potential stall warning

as well as airspeed and altitude disagreement

messages.

As the flight progressed, MCAS activated based on faulty data from

the aircraft’s external AOA sensor over 20 times, which led to loss of control of

the aircraft.

Notably, the accident aircraft experienced unintended MCAS activation the day

before with a different outcome. Shortly after departure on October 28, 2018,

Lion Air Flight 043, traveling from Denpasar to Jakarta, experienced a series of

failures similar to that of Lion Air 610. During the incident, the flight crew

successfully counteracted MCAS and used the stabilizer trim

cutout switches

to effectively “turn off” MCAS (see figure 10). The flight crew continued to Jakarta,

but with continuous cockpit alerts for the duration of the flight.

Specifically, the stick shaker warning was activated, an alert that warns the flight crew when the aircraft is close to a

wing stall condition.

These messages indicate that the airspeed and altitude data being reported by the Captain’s instruments and the

First Officer’s instruments do not match.

Trim systems help minimize a pilot’s workload by aerodynamically assisting in the movement and position of the

flight control surfaces. These systems can be manipulated manually but typically feature an electrically powered

system to assist pilots.

The stabilizer trim cutout switches remove power from the stabilizer trim motor when positioned to cutout.

AV2020037 30

Figure 10. 737 MAX Stabilizer Trim Cutout Switches

The stabilizer trim cutout switches are marked in the red circle above.

Source: Boeing

The Indonesian accident investigators found that the left AOA sensor on the 737

MAX 8 was replaced prior to Lion Air 043, but this replacement sensor

subsequently reported faulty data. Investigators were unable to determine if

required testing of the replacement sensor had been performed properly.

MCAS was designed to rely on a single AOA sensor, making it vulnerable to

failure from a sole source of erroneous input. Further, upon landing, the flight

crew did not fully report all of the issues experienced, making a complete

evaluation by maintenance technicians difficult.

Two types of testing on the replacement sensor were required: (1) functional testing of the replacement sensor by

the air carrier upon installation and (2) maintenance testing performed by the repair station prior to providing the

part to Lion Air. FAA has since revoked the vendor’s repair station certificate.

AV2020037 31

November 4–7, 2018

FAA conducts initial risk analysis following the Lion Air crash.

FAA conducted an initial review of events through its Continued Operational

Safety Program.

The review included a quantitative analysis of the service

history of the Boeing 737 MAX 8. Based on this review, FAA determined that

while urgent

action was necessary, the ongoing risk did not meet the threshold

for aircraft grounding. This analysis was the basis for issuing an Emergency AD.

November 6, 2018

Boeing issues bulletin to operators regarding the 737 MAX 8 and 737 MAX 9.

The bulletin informed 737 MAX 8 and MAX 9 operators that erroneous AOA data

could result in uncommanded nose-down movement of the aircraft and that this

action can repeat until the related system is deactivated. The bulletin emphasized

pilot procedures to perform, including returning the aircraft to a neutral trim

position, following the runaway stabilizer checklist, using the appropriate

switches (e.g., figure 10 above) to remove power from the related system, and

using manual trim once they turned the stabilizer off. The bulletin further

reminded pilots that they can experience additional indications and effects,

including but not limited to altitude and airspeed disagreement alerts.

November 7, 2018

FAA issues Emergency Airworthiness Directive.

One day after Boeing’s bulletin, FAA issued an AD

to all air carriers operating

the 737 MAX 8 and MAX 9. The AD identified that if the aircraft’s AOA sensor fails

or sends erroneous data to the flight control system, there is potential for

repeated nose-down movement that could lead to difficulty controlling the

airplane. The AD required all owners and operators of the 737 MAX 8 and MAX 9

aircraft to—within 3 days of the receipt of the AD—revise the Airplane Flight

Manual to provide flight crews with procedures to follow under certain conditions

that would counteract the aircraft’s nose-down movements. While neither the

bulletin nor the AD specifically named MCAS, Boeing issued a message to

operators of the aircraft on November 10, 2018, with a brief MCAS description.

FAA’s Continued Operational Safety Program is a data-driven program intended to manage risks associated with

specific aircraft fleets. It uses qualitative and quantitative analysis to determine the appropriate course of action

following potential safety events.

FAA uses a risk model with recommended levels of response including the probability of individual injury per flight

hour, such as pursuing immediate actions (1 fatal injury in 1 million flight hours) and grounding the aircraft (1 fatal

injury per 100,000 flight hours).

FAA issues ADs to aircraft owners and operators and transmits ADs to foreign aviation authorities. Emergency AD

2018-23-51 was issued on November 7, 2018.

AV2020037 32

November 28, 2018

FAA completes a risk analysis of the Lion Air Flight 610 accident.

FAA completed its initial Continued Operational Safety risk analysis and

subsequent review by the Corrective Action Review Board.

The risk analysis

results supported the original decision to issue the Emergency AD and stated that

additional action was required to further reduce risk. Specifically, the Board found

that the uncorrected individual risk

was 2.68 fatalities per 1 million flight hours.

This exceeds FAA’s Transport Airplane Risk Analysis Methodology (TARAM)

risk

guidelines of 1 fatality per 10 million flight hours.

Based on the risk analysis, Boeing proposed and FAA accepted a redesign of

MCAS software that would include additional safeguards against unintended

MCAS activation. FAA completed an additional analysis on December 12, 2018,

for the risk post-implementation of the AD. This analysis determined a risk of

about 15 accidents occurring over the life of the entire 737 MAX fleet if the

software fix was not implemented.

FAA’s risk analysis also indicated that the AD

mitigated the risk sufficiently enough to allow continued aircraft operation for a

limited period of time, until July 2019, while the software fix was being developed

and implemented on the existing fleet. As a result of the Lion Air accident, Boeing

agreed to begin developing software design changes to MCAS. The initial

proposal for the software fix would revise MCAS to compare data from both AOA

sensors and limit its ability to activate multiple times.

January 9, 2019

FAA begins review of MCAS certification process.

In January 2019, FAA initiated an internal review of the original MCAS certification

process. This was the first time FAA performed its own detailed analysis of MCAS,

and according to several FAA certification engineers, it was also the first time that

they were presented with a full picture of how MCAS worked. This review resulted

in documentation that was never finalized.

In the draft, FAA did not find any

non-compliances, but the Agency noted Boeing’s document traceability and

The Corrective Action Review Board is a panel of FAA experts who formally recommend the action to be taken,

which could include the issuance of an AD or grounding a specific model or fleet of aircraft.

Uncorrected individual risk is the probability of individual fatal injury per flight hour if no action is taken to address

an identified condition.

FAA’s TARAM handbook outlines a process for determining the numerical risk associated with the continued

operation of passenger carrying aircraft, and guidance for identifying unsafe conditions and corresponding regulatory

actions.

This figure of 15 accidents assumes an estimated fleet size of 4,800 aircraft in operation; there were approximately

250 aircraft in operation at the time of the Lion Air accident.

This documentation included both required supervision records and a draft report. According to FAA management,

the report was going through management review and comment at the time of the Ethiopian accident, at which time

the Agency considered it overtaken by events.

AV2020037 33

clarity of explanations were lacking in its revisions to MCAS and other system

certification documents. FAA’s post-accident review determined that an

independent reviewer would not have been able to effectively review the safety

assessment as a standalone compliance document or understand the full system

functionality and linkage with other systems and functions.

February 13, 2019

FAA and Boeing formally agree to a schedule for implementation of the MCAS

software fix.

Based on the analysis performed in December 2018, FAA determined that Boeing

and operators had until July 2019 to develop and implement the MCAS software

update in order to remain within the allowable risk guidelines contained in the

TARAM Handbook.

FAA and Boeing agreed to an implementation plan to meet

that date. Under the agreement, Boeing would develop the software update by

April 12, 2019; FAA would issue an AD requiring implementation of the new

software by April 19, 2019; and operators would have until June 18, 2019, to

install the software. According to FAA, the Agency calculated these milestones

based on the best information and data available at the time about the aircraft

and the Lion Air accident.

March 10, 2019

Ethiopian Airlines Flight 302 crashes, resulting in 157 fatalities.

Just over 4 months after the Lion Air Flight 610 crashed into the Java Sea,

Ethiopian Airlines Flight 302 crashed shortly after departing Addis Ababa Bole

International Airport, resulting in 157 fatalities, including 8 Americans. The interim

accident report

found that shortly after takeoff, the left AOA sensor failed.

MCAS again activated multiple times based on erroneous AOA data, resulting in

uncommanded nose-down movement of the aircraft. According to the report, the

flight crew used the stabilizer trim cutout switches. However, according to the

interim report, flight data from the investigation indicate the crew subsequently

returned power to the stabilizer trim system. This was not in compliance with

FAA’s Emergency AD, which states that the stabilizer trim cutout switch remain

set to the cutout position for the reminder of the flight. Ultimately, the crew lost

control of the aircraft. The interim investigation results show that the pilots did

not reduce power from takeoff thrust during the duration of the flight, resulting

in excessive speed. This can lead to aerodynamic forces that exceed the ability of

the flight crew to counteract these forces through manual adjustments.

TARAM Handbook and FAA Policy Statement. PS-ANM-25-05. November 4, 2011.

On March 9, 2020, the Federal Democratic Republic of Ethiopia, Ministry of Transport, Aircraft Accident

Investigation Bureau, released Interim Investigation Report of accident 737-8 MAX ET-AJV, ET-302.

AV2020037 34

Timeline of Concurrent FAA Organization

Designation Authorization Oversight Actions and

Events

During the same timeframe of the 737 MAX 8 certification and the Lion Air

accident, Boeing, FAA, and our office were identifying some significant

problems—although not specific to the 737 MAX 8—with the Boeing ODA, as

well as FAA’s ODA oversight. These included undue pressure on ODA unit

members, quality and timeliness of certification documentation, and the

effectiveness of FAA oversight.

2013–2019

Boeing ODA self-audits and surveys identify employee concerns about undue

pressure.

In two ODA self-audits—one in the Seattle, WA, area (2013) and another in

Charleston, SC (2014), Boeing identified employee concerns related to undue

pressure within the ODA. The term “undue pressure” describes situations in which

an ODA unit member or other designee—i.e., a company employee working on

behalf of FAA—faces conflicting non-ODA duties or interference from other

company or organizational elements regarding how to effectively administer

pertinent regulations.

Subsequently, in 2016, Boeing conducted an undue pressure survey of its ODA

unit members, obtaining 523 responses as of November 2016. While 97 percent

of the respondents agreed that they understood the process for reporting undue

pressure, almost 40 percent had encountered situations where they perceived

potential undue pressure, and almost a quarter of respondents had experienced

undue pressure beyond their direct reporting structure while performing their

ODA function. Further, respondent comments included common themes such as

pressure from high workloads, confusion and potential undue pressure due to

the dual roles of a unit member,

and a desire for the company to share

information about other undue pressure cases to help other unit members learn

and understand from those cases.

According to FAA, Agency discussion with Boeing and analysis of the survey

results indicated there may have been a distinction between “potential” undue

pressure and undue pressure that would rise to the level of formal reporting,

The Boeing ODA has nearly 1,500 personnel; however, ODA administrators and unit members perform those duties

only part-time. The same engineer can work for the company on a particular design and then approve that same

design as an ODA unit member. Boeing ODA managers who administer the program also have concurrent roles

within Boeing.

AV2020037 35

which was not differentiated in the survey. Interviews of ODA unit members

found that all formally reported instances of undue pressure were satisfactorily

addressed. However, according to the Agency, FAA observations indicated a need

for further oversight of the undue pressure systems and processes.

Over the course of 2018 and 2019, the Boeing ODA completed seven internal

audits that covered the undue pressure reporting process for ODA unit members

at selected Boeing facilities—not specific to the MAX. None of the audits found

non-conformities (e.g., violations of FAA regulations) related to undue pressure,

and the reports remarked that the process for reporting concerns about undue

pressure was well communicated and supported. However, the ODA’s self-audits

reported that while unit members in interviews demonstrated awareness and

knowledge of the undue pressure reporting process, one audit noted a

perception of “inadequate protection from actions by leadership outside of

ODA.” Although not a formal audit finding, another internal audit noted a

“general lack of confidence that the [undue pressure reporting] process would

reach a satisfactory conclusion and/or protect the Unit Members.”

October 15, 2015

DOT OIG issues report citing issues with staffing and processes for ODA

oversight.

While delegation is an essential part of meeting FAA’s certification goals, our

office has reported since 2011

that the Agency faces challenges in overseeing

ODA companies, including Boeing. For example, in October 2015, during the

timeframe of the Boeing 737 MAX 8 certification, we reported

that FAA’s

oversight of ODA program controls was not systems- and risk-based,

recommended by an aviation rulemaking committee.

Instead, FAA’s oversight

was more focused on individual engineering products and areas that we

determined were low risk. We also found that FAA lacked a comprehensive

process for determining staffing levels needed to provide ODA oversight and that

FAA did not conduct sufficient oversight of ODA personnel who performed

certification work at companies that supply components to manufacturers. We

made nine recommendations aimed at improving FAA’s staffing and oversight of

the ODA program.

FAA Needs To Strengthen Its Risk Assessment and Oversight Approach for Organization Designation Authorization

and Risk-Based Resource Targeting Programs (OIG Report No. AV2011136), June 29, 2011.

FAA Lacks an Effective Staffing Model and Risk-Based Oversight Process for Organization Designation Authorization

(OIG Report No. AV2016001), October 15, 2015.

Systems-based oversight shifts from focusing on individual project engineering work to holistically assessing

whether ODA companies have the people, processes, procedures, and facilities in place to produce safe products, thus

allowing FAA to focus its oversight on the highest-risk areas, such as new, innovative aircraft designs.

Aircraft Certification Process Review and Reform Aviation Rulemaking Committee, a joint FAA and industry group,

formed in response to a congressional mandate to study the aircraft certification process.

AV2020037 36

FAA has since addressed most recommendations from our report, but the Agency

has not yet implemented a risk-based approach to ODA oversight as we

recommended. This approach would allow FAA to assess the greatest risks and

target its oversight accordingly. FAA’s current plan is to implement its new

system for ODA oversight by December 2020.

During our current review, we found that engineers in FAA’s Boeing oversight

office still face challenges in balancing certification and oversight responsibilities.

While we have not found any evidence of an inappropriately close relationship

between FAA and Boeing to date, some FAA personnel expressed concern that

FAA executives are too deferential to Boeing.

2015–2019

FAA and Boeing sign a Settlement Agreement, and Boeing pays a $12 million

fine related to ongoing ODA manufacturing and certification issues.

During the same time period as the MAX 8 certification, FAA was performing

ODA oversight activities and enforcement actions related to manufacturing and

certification issues at Boeing, such as documentation quality, timeliness, and

corrective actions. In culmination of these efforts, FAA and Boeing signed a

Settlement Agreement on December 18, 2015, wherein Boeing agreed to take

actions in specified regulatory compliance areas and acknowledged obligations

to meet performance metrics. These actions would resolve allegations

documented in 13 FAA Enforcement Investigative Reports (EIR) spanning from

2009 through 2015.

All 13 EIRs cited violations of Boeing’s approved production, delegation, and

certification systems. Each EIR also cited violations of more specific regulations,

such as those governing completed aircraft conformity, fuel tank flammability,

compliance plans, and compliance with ODA procedures. Although not

specifically tied to the 737 MAX 8 aircraft, these allegations pertained to issues

regarding the quality of ODA certification documents and Boeing’s processes to

identify the root causes of its non-compliances

and establish corrective actions

to resolve them.

Upon signing the agreement, Boeing paid a $12 million civil penalty to FAA and

could face civil penalties up to $24 million if it fails to meet the settlement

agreement commitments by December 31, 2020.

Root causes are the contributory or initiating underlying causal factors of a nonconformity or undesirable event. A

causal factor is considered the root cause if its removal from the event sequence prevents the undesirable event from

recurring.

AV2020037 37

Between February 2017 and March 2019, FAA initiated 17 ODA oversight

activities

(called supervision records) related to the compliance findings in the

2015 Settlement Agreement. FAA deemed six (35 percent) of the activities to be

unsatisfactory, related to incomplete information and/or insufficient justification

provided in certification project documentation.

According to FAA and Boeing officials, the company is still working on improving

its documentation and processes, particularly in the area of identifying and

resolving root causes to prevent non-conformances and non-compliances from

recurring.

November 21, 2018

FAA initiates a formal compliance action against Boeing related to ODA

oversight.

During the same time period of the Lion Air accident, FAA was in the process of

completing its oversight related to potential undue pressure on engineering unit

members, not directly related to the 737 MAX. In November 2018, FAA initiated a

formal compliance action

against Boeing, citing five engineering unit members

who had conveyed to FAA instances of interference or conflicting duties with

their unit member roles. According to FAA, one of these five individuals reported

the instances of undue pressure through the formal Boeing process for

resolution.

In subsequent months, Boeing requested three extensions from FAA before

providing its response to the compliance action, including a corrective action

plan. FAA did not accept Boeing’s response to this compliance action. FAA also

issued two separate letters of investigation

in June 2019 and March 2020

against Boeing, related to potential undue pressure of unit members. FAA did not

accept Boeing’s response to the June 2019 letter of investigation and is currently

evaluating that letter of investigation and the formal compliance action together.

The Agency is still awaiting Boeing’s response to the more recent March 2020

letter of investigation.

FAA ODA oversight employees actually initiated 26 of these supervision records, but only 17 were accepted upon

review by FAA management. Of the 26 records initially submitted, 5 were deleted, 3 were rejected, and 1 had not yet

been reviewed as of July 25, 2019.

In contrast to a legal enforcement action, such as a civil penalty, compliance actions allow a manufacturer to

address a non-compliance in accordance with a corrective action plan agreed upon with FAA. According to FAA, an

insufficient response to a compliance action can result in enforcement action.

A Letter of Investigation, as part of FAA compliance and enforcement program, serves the dual purposes of

notifying an apparent violator that they are under investigation for a potential violation and providing the factual

details about the activities being investigated. It also gives the apparent violator an opportunity to provide input and

respond to the Agency. According to FAA, an insufficient response to a Letter of Investigation can result in

enforcement action.

AV2020037 38

Conclusion

FAA is charged with overseeing the safety and certification of all civilian aircraft

manufactured and operated in the United States. The tragic accidents in 2018

and 2019 involving the Boeing 737 MAX 8 aircraft have raised important

questions about FAA’s certification process, including its oversight of the ODA

program. The accidents, including FAA’s response following the Lion Air crash,

have also drawn attention to the Agency’s processes for determining certification

basis, assessing pilot training needs, and conducting risk analyses. While

investigations and related reviews are still ongoing, FAA’s sustained management

attention will be essential in identifying and monitoring the highest-risk and

safety-critical areas of aircraft certification, while also working to restore public

confidence in its aircraft certification processes.

Recommendations

Given that we have an open recommendation for FAA related to ODA and that

we are planning additional analysis of FAA’s certification process and the use of

the ODA program for the 737 MAX 8 aircraft, we are not making

recommendations in this report. In our 2015 report, we recommended FAA

develop and implement risk-based tools to aid ODA team members in targeting

their oversight. By December 31, 2020, FAA plans to implement a risk-based

approach to ODA oversight.

The data gathered for this report are informational and meant to be responsive

to the Secretary’s request. We will report further on FAA’s oversight of the

certification process, ODA, and other related matters in future reports.

Agency Comments and OIG Response

We provided FAA with our draft report on April 28, 2020, and received its

response on June 8, 2020, which is included as an appendix to this report. As our

report did not contain recommendations, no further actions are required.

Exhibit A. Scope and Methodology 39

Exhibit A. Scope and Methodology

We conducted this performance audit between April 2019 and April 2020 in

accordance with generally accepted Government auditing standards as

prescribed by the Comptroller General of the United States. Those standards

require that we plan and perform the audit to obtain sufficient, appropriate

evidence to provide a reasonable basis for our findings and conclusions based on

our audit objectives.

We believe that the evidence obtained provides a reasonable basis for our

findings and conclusions based on our audit objectives. This report is in response

to the Secretary’s request to compile an objective and detailed factual history of

the activities that resulted in the certification of the 737 MAX 8 and is the initial

product to be issued related to FAA’s certification of the Boeing 737 MAX.

To determine the reliability of the data, we compared dates regarding Boeing 737

MAX certification documentation received from both FAA and Boeing and

obtained source documentation to confirm and resolve discrepancies from

respective presentations. We also sought and obtained source documentation to

verify information obtained from testimonial evidence. In addition, we assessed

the completeness and integrity of FAA’s ODA oversight records by reviewing the

content and accuracy of the data and determining FAA’s processes for assessing

data quality.

To obtain detailed, factual information regarding FAA’s aircraft certification

process and the historical certification of Boeing’s 737 MAX, we met with FAA

aircraft certification officials in both Washington, DC, and Oklahoma City, OK, to

discuss the evolution of FAA’s certification and ODA policies and guidance. We

also collected current and historical ODA policy and guidance documents and

internal policy office analyses, including data tracking metrics for Boeing’s ODA.

We received multiple briefings from FAA’s Aircraft Certification Service, System

Oversight Division and Compliance and Airworthiness Division, as well as the

Flight Standards Service, Aircraft Evaluation Group, located at the Northwest

Mountain Regional Office. We conducted interviews of FAA safety inspectors;

flight test and control engineers; standards staff engineers; and certification,

oversight, and flight test management personnel. We reviewed and analyzed

certification plans and associated deliverables, issue papers, internal

correspondence, internal safety analyses conducted during the certification

process and following the 2018 and 2019 accidents, and flight test documents

pertaining to the Boeing 737 MAX. We also interviewed a National Air Traffic

Controllers Association representative for FAA aircraft certification engineers to

look at issues raised over the course of the MAX’s certification.

Exhibit A. Scope and Methodology 40

We analyzed the 2015 Settlement Agreement between FAA and Boeing and

collected and reviewed supervisory records, pilot reports, ODA audits, and

compliance and enforcement actions that FAA initiated on Boeing during the

timeframe of the MAX’s certification. We also collected information related to

recent compliance actions regarding undue pressure of Boeing ODA employees.

We visited Boeing facilities in Everett, Renton, and Seattle, WA, and interviewed

Boeing management about the Boeing 737 MAX’s certification. We collected

further documentation from Boeing regarding certification plans, internal system

safety analyses, MCAS-specific requirements and testing documents, internal

flight test reports, and updates regarding return-to-service actions and MCAS

software revisions. We also interviewed ODA management and collected and

reviewed internal ODA procedure manuals and self-audits. Interviews of Boeing

certification personnel were limited in scope because of liability concerns raised

by Boeing. Individual interviews of Boeing ODA staff to obtain information about

undue pressure and other climate issues were conducted within agreed-upon

parameters with Boeing, such as OIG not asking the two ODA staff interviewed

about specific certification decisions or certification programs such as the 737

MAX.

Finally, we had several coordination meetings with the National Transportation

Safety Board over the course of our audit, to both avoid duplication of effort in

our respective reviews and to also receive updates on the status of its accident

investigations.

Exhibit B. Organizations Visited or Contacted 41

Exhibit B. Organizations Visited or Contacted

Federal Aviation Administration

Aircraft Certification Service:

System Oversight Division

Boeing Aviation Safety Oversight Office, Des Moines, WA

Boeing Certificate Management Office, Des Moines, WA

Compliance and Airworthiness Division

Northwest Flight Test Section, Des Moines, WA

Seattle Aircraft Certification Office, Des Moines, WA

Policy and Innovation Division

Transport Standards Branch – Des Moines, WA

Certification Procedures Branch – Washington DC

Delegation and Organizational Procedures Branch – Oklahoma

City, OK

Flight Standards Service (AFX):

Seattle Aircraft Evaluation Group, Des Moines, WA

Other Organizations

Boeing Commercial Airplanes

Everett, WA

Renton, WA

Seattle, WA

National Air Traffic Controllers Association

National Transportation Safety Board

Exhibit C. Glossary of Terms 42

Exhibit C. Glossary of Terms

Term Definition

AC Advisory Circular Advisory Circulars are guidance documents produced by FAA to inform and

guide entities within the aviation industry, as well as the general public, and

describe actions or advice that FAA expects to be implemented or followed.

AD Airworthiness Directive ADs are legally enforceable rules issued by FAA to correct an unsafe condition in

a product. 14 CFR Part 39 defines a product as an aircraft, aircraft engine,

propeller, or appliance.

AEG Aircraft Evaluation

Group

A group in FAA’s Flight Standards Service tasked with determining the

appropriate types of training for aircraft that are undergoing evaluation for an

ATC.

AFM Airplane Flight Manual An Airplane Flight Manual’s primary purpose is to provide an authoritative

source of information considered necessary for safely operating the airplane.

AFMs have specific information that must be provided to satisfy airworthiness

regulations.

AOA Angle of Attack The difference between the pitch angle (nose direction) of the airplane and the

angle of the oncoming wind. AOA sensors measure the angle between an

airplane’s wing and the oncoming air.

ATC Amended Type

Certificate

An ATC is issued by FAA when the holder of a type certificate receives FAA

approval to modify an aircraft design from its original design. An ATC approves

not only the modification but also how that modification affects the original

design.

BASOO

Boeing Aviation Safety

Oversight Office

Provides oversight of designee authority granted to Boeing.

COS Continued Operational

Safety

A data-driven, risk-based approach for safety assurance and safety risk

management.

FAA

Federal Aviation

Administration

The Agency responsible for overseeing numerous aviation activities designed to

ensure the safety of the flying public.

FCC Flight Control

Computer

The component of digital flight control software that provides several functions

integral to flight, including autopilot, flight director, and speed trim.

FCOM

Flight Crew Operations

Manual

The FCOM contains operations information and provides the necessary

operating limitations, procedures, performance, and systems information the

flight crew needs to safely and efficiently operate the aircraft.

Exhibit C. Glossary of Terms 43

Term Definition

FSB Flight Standardization

Board

FAA typically establishes an FSB when certificating large jet or propeller aircraft.

One of the FSB’s mandates is to develop training objectives for normal and

emergency procedures and maneuvers.

JATR Joint Authorities

Technical Review

The JATR is a team consisting of representatives of regulators from 10 civil

aviation authorities that was chartered by FAA on June 1, 2019, to examine the

Agency’s certification of the 737 MAX 8. The JATR issued a report on October

11, 2019.

JOEB

Joint Operational

Evaluation Board

A multi-regulatory body that conducts a multi-day session with global

regulatory and airline pilots to validate training requirements.

MDR Master Differences

Requirements

Specifies the highest training and checking difference levels between a pair of

related aircraft derived from the Differences Tables.

MCAS

Maneuvering

Characteristics

Augmentation System

Flight control law implemented on the 737 MAX to improve aircraft handling

characteristics and decrease pitch-up tendency at elevated angles of attack.

MoC Means of Compliance The means by which an applicant shows compliance with the flight

requirements for an airworthiness or type certificate.

NTSB

National Transportation

Safety Board

The NTSB conducts independent accident investigations, advocates safety

improvements, and decides pilots’ and mariners’ certification appeals.

ODA Organization

Designation

Authorization

FAA created the ODA program in 2005 to standardize its oversight of

organizational designees (e.g., aircraft manufacturers) that have been approved

to perform certain functions on the Agency’s behalf, such as determining

compliance with aircraft certification regulations.

S&MF

Single and Multiple

Failure

Boeing uses the S&MF document to analyze the probability of potential failures

of key systems and equipment, the probability that those failures will interact,

and the impact of multiple failures on continued safe flight and landing.

SSA

System Safety

Assessment

An assessment of the process to identify and classify failure conditions and

ensuing means for regulatory compliance.

TARAM Transport Airplane Risk

Analysis Methodology

Outlines a process for calculating risk associated with continued-operational-

safety (COS) issues in the transport-airplane fleet. It explains how to use such

risk-analysis calculations when making determinations of unsafe conditions and

selecting and implementing corrective actions.

TC Type Certificate

An approval document issued by FAA that states a specific aircraft model is

compliant with airworthiness regulations.

Exhibit C. Glossary of Terms 44

Term Definition

TIA Type Inspection

Authorization

Issued after FAA reviews the applicant’s test results package, the TIA authorizes

official conformity, airworthiness inspections, and ground and flight tests

necessary to fulfill TC certification requirements.

Exhibit D. Major Contributors to This Report 45

Exhibit D. Major Contributors to This Report

ROBIN KOCH PROGRAM DIRECTOR

MARSHALL JACKSON PROGRAM DIRECTOR

STEFANIE MCCANS PROJECT MANAGER

CHRISTOPHER FRANK PROJECT MANAGER

MELISSA PYRON SENIOR AUDITOR

ANDREW FARNSWORTH SENIOR ANALYST

KEVIN MONTGOMERY SENIOR ANALYST

HENNING THIEL SENIOR ANALYST

AIESHA MCKENZIE SENIOR ANALYST

JASON LEWIS ANALYST

RACHEL MENCIAS AUDITOR

GRACE ITA-CICCHELLI ANALYST

SETH KAUFMAN DEPUTY CHIEF COUNSEL

AUDRE AZUOLAS SENIOR TECHNICAL WRITER

SHAWN SALES VISUAL COMMUNICATIONS SPECIALIST

MAKESI ORMOND STATISTICIAN

Appendix. Agency Comments 46

Appendix. Agency Comments

U.S. Department of

Transportation

1200 New Jersey Avenue, S.E.

Washington, D.C. 20590

Office of the Secretary of

Transportation

MEMORANDUM

Date: June 8, 2020

Subject: INFORMATION: Management Response to Office of Inspector

General (OIG) Draft Report on FAA’s Oversight of Boeing 737 MAX

Certification

From: Steven G. Bradbury

General Counsel (and performing the

functions and duties of Deputy Secretary)

To: Howard R. Elliott

Acting Inspector General

The Department of Transportation (DOT) appreciates the opportunity to review the draft report

by the Office of Inspector General (OIG) documenting the timeline of the certification of the

Boeing 737 MAX aircraft, the use of Organization Designation Authorization (ODA) programs

at the Federal Aviation Administration (FAA), and FAA’s actions after the Lion Air Flight 610

and Ethiopian Airlines Flight 302 accidents. The Department’s top priority is safety. OIG’s

review will help FAA to better understand some of the factors that may have contributed to the

crashes and ensure these types of accidents never occur again.

Background on Ongoing MAX Re-certification Process

The Department of Transportation (DOT) and FAA continue to extend our deepest sympathy and

condolences to the families of the victims of Lion Air Flight 610 and Ethiopian Airlines Flight

302. We honor the memory of those 346 lives by striving for the highest possible margin of

safety in the global aviation system.

FAA’s aviation safety professionals have our unequivocal support in carrying out their critical

mission. They are following a thorough process for returning the 737 MAX to service—a

process that is not driven by a timeline, but by safety. As Administrator Dickson has testified,

“The FAA is continuing to follow a data-driven, methodical analysis, review, and validation of

the modified flight control systems and pilot training required to safely return the 737 MAX to

commercial service.” He has directed FAA employees to take whatever time is necessary to

Appendix. Agency Comments 47

complete their work. In addition, FAA will continue to coordinate with foreign airworthiness

authorities around the world as work to return the 737 MAX to service proceeds.

FAA’s Approach to Certifying the MAX Series

The timeline prepared by OIG reveals some strengths in FAA’s aircraft certification process, as

well as areas for improvement. This and other reviews, both completed and ongoing, will inform

important reforms of FAA’s aircraft certification process. Although OIG determined that FAA

followed its certification process for the MAX, OIG’s review also makes clear that FAA’s

certification of the 737 MAX was hampered by a lack of effective communication, both between

Boeing and FAA and within FAA, which led to an incomplete understanding of the scope and

potential safety impacts of changes to the flight control system. For example, OIG noted that

during the original certification process, “key FAA certification engineers and personnel

responsible for approving the level of airline pilot training were unaware of the revision to [the

Maneuvering Characteristic Augmentation System (MCAS)].” FAA’s certification process relies

on receiving complete, candid information from manufacturers. The agency will be taking

further steps to ensure integrity and transparency with regard to information sharing,

assumptions, and validation, all of which are integral to the overall certification system.

Additionally, FAA anticipates strengthening coordination among the lines of business with

certification responsibilities, as well as enhancing its human factors, flight controls, and system

safety expertise to address weaknesses that led to an incomplete understanding of MCAS prior to

certification.

FAA’s ongoing work to improve its certification process includes moving toward holistic review

and oversight from initial application to final certification, as well as coordinating a flexible

information flow throughout the oversight process. It also includes promoting an environment

where the proactive self-disclosure of errors is expected and appreciated, and where the reporting

of safety issues is encouraged. Safety Management Systems (SMS) for all industries involved in

the aerospace system and “Just Culture” concepts that allow for the consideration of honest

mistakes and incentivize openness and transparency will help achieve these goals.

Use of Organization Designation Authorization (ODA)

FAA is currently in the process of developing a new policy to ensure appropriate FAA oversight

of ODA programs. Delegation in the aviation industry has existed in some form since the 1920s,

and since 1958 for aircraft. Congress established the current ODA program, which sought to

streamline aspects of the certification process, as part of the FAA Modernization and Reform Act

of 2012 (Pub. L. 112-95, sec. 312, codified at 49 U.S.C. § 44704(e)). As Administrator Dickson

testified in December, the ODA “construct is based on trust . . . it’s a privilege . . . it’s not a

right.” FAA’s new policy will be based on that theme—building upon the successes of ODA

while ensuring the appropriate level of FAA oversight based on the risks associated with each

ODA’s authorized functions, the size and complexity of the ODA’s organization, any history of

undue influence on ODA decision-making, and other performance and risk factors. FAA will

also base the policy on recommendations from the evaluations currently underway by OIG and

others, such as the Expert Review Panel recently formed per section 213 of the FAA

Appendix. Agency Comments 48

Reauthorization Act of 2018. FAA expects to have the draft policy prepared for public comment

in the coming months.

Additionally, FAA has established the Organization Designation Authorization Office within the

Aviation Safety Organization. This office will provide a system-level focus on multiple areas,

including ODA utilization, establishment of ODA limitations, ODA oversight, and performance

of ODAs. By looking at the ODA system from an integrated certification and operational

perspective, this office will support standardized outcomes and drive improvement across all our

ODAs while coordinating national program policy. FAA is currently in the process of

operationalizing this office with the goal of having permanent staff by the end of calendar year

2020. Currently, the office is developing a detailed implementation plan to further operationalize

the office while defining actions needed to address continuous improvement of the ODA

Program.

Post-Accident

Within a matter of days following the Lion Air crash, FAA issued an emergency Airworthiness

Directive (AD) in response to the accident. The AD reminded pilots how to handle a runaway

stabilizer scenario, since an unintended MCAS activation was understood to manifest to pilots as

a runaway stabilizer. FAA made the decision to issue the emergency AD based on the

information available at the time. After the accident, FAA also initiated a review of the original

MCAS certification process and began certification work on the initial changes Boeing proposed

to address preliminary concerns about the 737 MAX. That work has been incorporated into

FAA’s ongoing recertification process.

Conclusion

The events noted on the OIG timeline are all important steps that are being evaluated with the

goal of identifying potential improvements to FAA’s risk-assessment methodology, development

of assumptions, decision-making, and information reliance. DOT looks forward to the results of

the OIG’s continued review including its recommendations to FAA later this year.

Department and FAA leadership appreciate the work of OIG, the Secretary’s Special Committee

to Review the FAA’s Certification Process for the 737 MAX, the National Transportation Safety

Board, the 737 MAX Technical Advisory Board, the Joint Authorities Technical Review, and the

accident investigation authorities to help FAA improve aviation safety both domestically and

abroad. FAA welcomes feedback from its international peers, intergovernmental partners,

governmental auditors, Congress, and industry experts. There will never be a risk-free mode of

transportation, but it is the dedication and hard work of aviation safety professionals within FAA

and throughout the industry that have made commercial aviation in the U.S. the safest mode of

transportation in the world.

We appreciate this opportunity to respond to the draft report. Please contact Madeline

Chulumovich at (202) 366-6512, with questions or if you require additional information.

Our Mission

OIG conducts audits and investigations on

behalf of the American public to improve the

performance and integrity of DOT’s programs

to ensure a safe, efficient, and effective

national transportation system.