This is the submitted version of the following journal - QUT ePrints

This is the submitted version of the following journal - QUT ePrints

QUT Digital Repository: http://eprints.qut.edu.au/ This is the submitted version of the following journal article:  Clothier, Reece A., Palmer, Jenni...

NAN Sizes 0 Downloads 0 Views

Recommend Documents

Accepted Version (PDF 463kB) - QUT ePrints
2. , and John Frazer. 1. 1. School of Design, Queensland University of .... Council House 2 (CH2) in Melbourne, the firs

lication in the following source: McGowan, Lee - QUT ePrints
McGowan, Lee. (2017). From banned to international glory, women's soccer has sown a rich field ... This Sunday, local te

Accepted Version (PDF 352kB) - QUT ePrints
Jan 6, 2011 - over five years and completing the design of a new heavy-lift launch vehicle by .... have societies and th

Accepted Version (PDF 182kB) - QUT ePrints
'unsellables'$were:$Cancel*Christmas$and$Increase*school*hours*adding*a*half6day*on*. Saturday.**Students$used$QUT's$dig

Accepted Version (PDF 964kB) - QUT ePrints
PLGA-Based Microparticles for the Sustained Release of. BMP-2. Giles T. S. Kirby. 1,. , Lisa J. White. 1. , Cheryl V. Ra

Draft Version (PDF 84kB) - QUT ePrints
Philip Graham. School of Communication. Faculty of Business. Queensland University of Technology. Email: [email protected]

This is the peer reviewed version of the following - Universität Stuttgart
Johannes E. M. N. Klein,[a] Gerald Knizia,[b] Burkhard Miehlich,[a] Johannes Kästner[b] und Bernd ..... Tolstikov, Russ

PDF (142kB) - QUT ePrints
in excess of 20% expected in 2007 in each global region (Kane, 2004; .... the top ten selling video games were based on

PDF 177kB - QUT ePrints
7 Singer & Friedlander Ltd v John D Wood & Co [1977] 2 EGLR 84. 8 Zubaida v Hargreaves [1995] 1 EGLR 127. 9 Singer & Fri

PDF (71kB) - QUT ePrints
(with 26.2 millions of skier visits) in North America and generated annual revenue over USD $2 billion. Four conglomerat

QUT Digital Repository: http://eprints.qut.edu.au/

This is the submitted version of the following journal article:  Clothier, Reece A., Palmer, Jennifer L., Walker, Rodney A., &  Fulton, Neale L. (2011) Definition of an airworthiness  certification framework for civil unmanned aircraft systems.  Safety Science.  © Copyright 2011 Please consult the authors.

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

                 

 

NOTICE:   this   is   the   author's   version   of   a   work   that   was   accepted   for   publication   in   Safety   Science.   Changes   resulting   from   the   publishing   process,   such   as   peer   review,   editing,   corrections,   structural   formatting,   and   other   quality   control   mechanisms   may   not   be   reflected   in   this   document.   Changes   may   have   been   made   to   this   work   since   it   was   submitted   for   publication.   A   definitive   version   is   to   be   published   in   Safety   Science:     Authors:  Clothier,  RA.,  Palmer,  JL.,  Walker,  RA.,  and  Fulton,  NL.   Title:   Definition   of   an   Airworthiness   Certification   Framework   for   Civil   Unmanned   Aircraft   Systems   (Article   in   Press,   Corrected   Proof)     Publication:  Safety  Science   Available  online:  5  March  2011   DOI:  http://dx.doi.org/10.1016/j.ssci.2011.02.004    

©  Copyright  2011  Reece  Clothier  

 

Page  1  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

       

TITLE:     Definition  of  an  Airworthiness  Certification  Framework  for  Civil   Unmanned  Aircraft  Systems    

AUTHORS:   Mr  Reece  A.  Clothier  1   Dr  Jennifer  L.  Palmer  2     Prof  Rodney  A.  Walker  1     Dr  Neale  L.  Fulton  3            

                                                                                                                                     Australian  Research  Centre  for  Aerospace  Automation  (ARCAA)   Queensland  University  of  Technology   22-­‐24  Boronia  Rd,  Eagle  Farm,  QLD  4009,  Australia   [email protected],  [email protected],  Ph:  +61(7)  3138  1403     2  Defence  Science  and  Technology  Organisation  (DSTO)   Air  Vehicles  Division,  Flight  Systems  Branch     506  Lorimer  Street,  Fishermans  Bend,  Melbourne,  VIC  3207,  Australia   [email protected],  Ph:  +61(3)  9626  8047,  Fax:  +61(3)  9626  7085     3  Commonwealth  Scientific  and  Industrial  Research  Organisation  (CSIRO)   CSIRO  Mathematics,  Informatics  and  Statistics   GPO  Box  664,  Canberra,  ACT  2601,  Australia   [email protected],  Ph:  +61(2)  6216  7058       1

©  Copyright  2011  Reece  Clothier  

 

Page  2  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

Abstract   The   development   of   effective   safety   regulations   for   unmanned   aircraft   systems   (UAS)   is   an   issue   of   paramount  concern  for  industry.  The  development  of  this  framework  is  a  prerequisite  for  greater  UAS  access   to  civil  airspace  and,  subsequently,  the  continued  growth  of  the   UAS  industry.  The  direct  use  of  the  existing   conventionally   piloted   aircraft   (CPA)   airworthiness   certification   framework   for   the   regulation   of   UAS   has   a   number  of  limitations.   The   objective   of   this   paper   is   to   present   one   possible   approach   for   the   structuring   of   airworthiness  regulations  for  civilian  UAS.  The  proposed  approach  facilitates  a  more  systematic,  objective  and   justifiable  method  for  managing  the  spectrum  of  risk  associated  with  the  diversity  of  UAS  and  their  potential   operations.   A   risk   matrix   is   used   to   guide   the   development   of   an   airworthiness   certification   matrix   (ACM).   The   ACM   provides   a   structured   categorisation   that   facilitates   the   future   tailoring   of   regulations   proportionate   to   the  levels  of  risk  associated  with  the  operation  of  the  UAS.  As  a  result,  an  objective  and  traceable  link  may  be   established  between  mandated  regulations  and  the  overarching  objective  for  an  equivalent  level  of  safety  to   CPA.  The  ACM  also  facilitates  the  systematic  consideration  of  a  range  of  technical  and  operational  mitigation   strategies.  For  these  reasons,  the  ACM  is  proposed  as  a  suitable  method  for  the  structuring  of  an  airworthiness   certification   framework   for   civil   or   commercially   operated   UAS   (i.e.,   the   UAS   equivalent   in   function   to   the   Part   21   regulations   for   civil   CPA)   and   for   the   further   structuring   of   requirements   on   the   operation   of   UAS   in   un-­‐ segregated  airspace.      

Keywords   Airworthiness,  Regulation,  UAS,  Unmanned  Aircraft,  Risk  Matrix      

1 Introduction  

4

The   development   of   an   effective   regulatory   framework   for   unmanned   aircraft   systems   (UAS )   is   a   major   concern  for  manufacturers  and  operators.  Civil  and  military  UAS  operations  are  currently  subject  to  restrictions   that   significantly   inhibit   their   flight   within   un-­‐segregated   civil   airspace   and   over   populated   areas.   These   restrictions  limit  both  the  military  and  non-­‐military  use  of  UAS.  A  greater  degree  of  operational  freedom  will   only   occur   through   the   development   of   a   framework   inclusive   of   regulations,   engineering   and   training   standards,  flight  rules  and  operational  practices  that  can  be  shown  to  deliver,  at  a  minimum,  a  level  of  safety   equivalent   to   that   currently   exhibited   by   civilian   conventionally   piloted   aircraft   (CPA).   This   requirement   is   referred  to  as  the  equivalent  level  of  safety  (ELOS)  objective  (JAA  2004;  OSD  2007;  CAA  2008).  The  particular   regulations  discussed  in  this  paper  relate  to  the  “airworthiness”  (ADF  2002)  of  the  system  being  operated.  A   brief  introduction  to  the  concept  of  airworthiness  and  its  regulation  is  presented  in  Section  §2.1.     Much   effort   is   being   devoted   to   the   definition   of   standards   specific   to   UAS   (e.g.,   the   specification   of   prescriptive   requirements   on   aspects   of   their   design,   maintenance,   manufacture   and   operation).   However,   little   consideration   has   been   given   to   how   these   standards   and   regulations   may   be   appropriately   applied   across   the   diversity   of   UAS,   their   operations   and   the   mitigation   strategies   widely   employed.   As   discussed   in   §2.2,   there   is   currently   no   consensus   on   the   specification   of   airworthiness   regulations   for   UAS.   The   default   proposal  is  to  apply  the  CPA  airworthiness  regulatory  framework  to  that  of  UAS  (Dalamagkidis  et  al.  2008b).   Such  an  “off-­‐the-­‐shelf”  (Clothier  et  al.  2008)  approach  is  implicitly  based  on  the  premise  that  the  application  of   “equivalent”   regulations   will   yield   an   ELOS,   despite   UAS   being   described   as   a   fundamentally   different   hazardous  paradigm.  Among  many  others,  Hayhurst   et  al.  (2006),  Dalamagkidis  et  al.  (2008b),  Clothier  et  al.   (2006,   2008)   and   DeGarmo   (2004)   identify   numerous   differences   between   UAS   and   CPA   that   challenge   the   suitability   of   the   off-­‐the-­‐shelf   approach.   These   include   differences   between   the   associated   risks,   design   philosophies,  applications,  operational  concepts,  and  supporting  business  cases  and  differences  between  the   social  and  political  attitudes  towards  the  two  technologies  (further  described  in  §2.3).  As  will  be  discussed  in   §2.3,   the   primary   risks   that   governed   the   development   of   the   CPA   airworthiness   regulatory   framework   are   different  to  those  that  should  be  used  to  guide  the  development  of  an  airworthiness  framework  for  UAS.  This   dissimilarity   can   result   in:   airworthiness   regulations   that   do   not   ensure   the   equitable   management   of   the   risks                                                                                                                                        The  plural  acronym  is  the  same  as  the  singular,  in  accordance  with  OSD  (2007)  Unmanned  Systems  Roadmap,   2007–2032.  Washington,  DC,  United  States  Department  of  Defense,  Office  of  the  Secretary  of  Defense.   4

©  Copyright  2011  Reece  Clothier  

 

Page  3  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

across   all   types   of   UAS   and   their   operations;   the   potential   over-­‐regulation   and   hence   imposition   of   unnecessary  costs  to  the  UAS  industry;  or  worse,  an  airworthiness  regulatory  framework  that  does  not  satisfy   the  objective  for  an  equivalent  level  of  safety.   It  has  been  concluded  that  UAS  “will  require  a  new  regulatory  framework  to  both  maintain  the  safety  of   the  national  airspace  system  and  to  enable  the  full  benefit  of  unmanned  aviation”  (Hayhurst  et  al.  2006).  The   objective  of  this  paper  is  to  present  a  practical  framework  for  the  effective  regulation  of  the  airworthiness  of   civil  UAS.  

1.1 Guiding  Principles   This  leads  to  the  question  as  to  what  are  the  properties  of  an  effective  regulation?  The  high-­‐level  objective   for   aviation   safety   regulations   is   to   ensure   the   safe   and   effective   operation   of   aircraft,   with   priority   on   the   former.   Increased   regulation   often   results   in   increased   costs   to   the   industry   and,   in   turn,   a   reduction   in   the   potential   benefits   available   to   society   (Fischhoff   et   al.   1978).   Regulations   should   therefore   be   defined   and   promulgated   proportionate   to   the   estimated   risks   associated   with   the   activity.   This   ensures   that   safety   objectives   can   be   verified   (e.g.,   assurance   that   the   regulations   satisfy   the   ELOS   objective)   and   that   the   regulatory-­‐costs  imposed  on  the  industry  are  warranted.  Under  this  premise,  a  justifiable  position  to  guide  the   specification  of  a  regulatory  framework  for  UAS  resides  in  an  understanding  of  the  risks  associated  with  their   operation.     5 To  summarise,  an  effective  regulatory  framework  for  UAS  should  be:       1. justifiable  (i.e.,  have  a  clear  basis  in  risk  and  traceability  to  the  ELOS  objective),     2. flexible   enough   to   accommodate   the   diversity   of   UAS   designs,   operations   and   mitigation   strategies,     3. systematic  in  its  management  of  the  risks  across  this  diversity,   4. objective   (i.e.,   the   underlying   methodology   should   be   independent   of   any   single   stakeholder’s   preferences),   5. practicable  in  its  implementation  (i.e.,  regulatory  authorities  should  have  a  workable  framework),   and   6. cognisant  of  the  costs  that  undue  regulations  impose  on  the  industry,  though  not  at  the  expense   of  the  objective  for  an  ELOS.     In  consideration  of  the  above  guidance,  a  quantified  risk  matrix  is  proposed  as  a  suitable  framework  for   guiding   the   structuring   of   regulations   for   the   airworthiness   of   UAS   and   their   integration   into   the   National   Airspace  System  (NAS).  The  foundational  concepts  of  the  risk  matrix  are  presented  in  Section  §3  in  the  context   of   developing   an   airworthiness   certification   framework   for   UAS.   The   output   is   a   systematic   and   flexible   framework   equivalent   in   regulatory   function   to   the   existing   civil   CPA   Part   21   regulation   (CASA   2003).   The   airworthiness  certification  matrix  (ACM)  developed  supports  both  an  approach  of  regulation  by  “safety-­‐target”   (Haddon  and  Whittaker  2002)  and  regulation  by  prescriptive  codes  of  requirements.   Within   the   structure   of   the   ACM,   the   composite   of   system   type   and   operational   environment   defines   airworthiness   categories.   Regulations   may   then   be   tailored   to   a   specific   airworthiness   category   through   the   quantification   of   measures   of   risk   relative   to   the   overarching   goal   for   an   ELOS.   Such   a   top-­‐down   tailoring   of   regulations   to   the   specific   joint-­‐categories   of   system   and   operation   establishes   the   scope   for   a   bottom-­‐up   hazard  analysis,  as  advocated  by  Hayhurst  et  al.  (2006)  and  comprehensively  described  by  Luxhøj  (2009).  An   example   of   the   top-­‐down   tailoring   of   regulations   is   described   in   Section   §3.7.1.   for   the   case   of   determining   Part  1309  regulations  for  UAS  that  are  equivalent  in  terms  of  regulatory  function  to  those  mandated  for  CPA   [e.g.,   Federal   Aviation   Administration   (FAA)   advisory   circulars   23.1309-­‐1D   (FAA   2009)   and   25.1309-­‐1A   (FAA   1988)].  An  expanded  and  generalised  risk  matrix  may  also  be  used  to  structure  other  dimensions  of  regulation,   including   the   operational   requirements   governing   the   integration   of   UAS   within   the   NAS.   This   application   is   briefly  discussed  in  Section  §4.   In   summary,   it   is   proposed   that   the   risk-­‐management   approach   described   in   this   paper   provides   a   justifiable,   flexible,   systematic,   objective   and   practical   method   for   structuring   the   regulation   of   UAS.   It   permits   the   incremental   development   of   regulations   and,   in   turn,   the   phased   integration   of   UAS   into   the   NAS.   The                                                                                                                                         The   definition   of   a   formal   evaluation   scheme   is   an   area   of   ongoing   research.   The   reader   is   referred   to   the   forthcoming  work  of  Michael  Nas  (2011),  Classifying  Unmanned  Aircraft  Systems:  Development  of  an  Objective   Framework  for  Evaluating  UAS  Classification  Schemes,  Murdoch  University,  WA  (unpublished).   5

©  Copyright  2011  Reece  Clothier  

 

Page  4  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

structuring  of  regulations  based  on  the  principles  of  a  risk  matrix  also  ensures  that  regulations  are  verifiable   against   the   overarching   objective   for   an   ELOS   and   that   any   impositions   due   to   regulations   are   a   justifiable   expense  to  the  industry.      

2 An  Airworthiness  Certification  Framework  for  UAS   The   precedence   for   the   establishment   of   international   regulations   governing   the   airworthiness   of   a   civil   aircraft   stems   from   the   Chicago   Convention   of   1944   (ICAO   2000).   Article   31   of   the   Convention   (ICAO   2000)   requires  aircraft  to  be  certificated  as  airworthy,  and  under  Article  33  these  certificates  must  be  recognised  as   valid  by  the  other  contracting  States  provided  they  are  equal  to  or  above  the  minimum  requirements  specified   in   the   Convention   (detailed   in   Annexes).   Article   8   of   the   Convention   stipulates   the   extension   of   these   requirements   to   UAS.   As   described   in   Annex   8   (ICAO   2005)   to   the   Convention,   the   objective   of   these   regulations  is  to  achieve,  “among  other  things,  protection  of  other  aircraft,  third  parties  and  property”.   There  are  several  definitions  for  the  concept  of  airworthiness,  the  most  comprehensive  of  which  identifies   the   key   components   of   airworthiness   regulations   and   is   provided   in   the   Australian   Defence   Force   (ADF)   instructions:     …a   concept,   the   application   of   which   defines   the   condition   of   an   aircraft   and   supplies   the   basis   for   judgement   of   the   suitability   for   flight   of   that   aircraft,   in   that   it   has   been   designed,   constructed,   maintained   and   is   expected   to   be   operated   to   approved   standards   and   limitations,   by   competent   and   approved   individuals,   who   are   acting   as   members   of   an   approved   organisation   and   whose   work   is   both   certified   as   correct  and  accepted  on  behalf  of  the  ADF.     p.  AL1,  ADF  2002  

2.1 Airworthiness  Certification   This   section   provides   a   brief   overview   of   airworthiness   regulations   for   civil   CPA.   For   further   detail   the   reader  is  referred  to  CASA  (2000),  FAA  (2004a)  and  Dalamagkidis  et  al.  (2009).   In   general,   a   Certificate   of   Airworthiness   (COA)   is   issued   for   an   individual   aircraft   if   it   meets   the   conditions   of   the   certification   of   its   type   (e.g.,   make   and   model)   against   prescriptive   requirements   stipulated   under   different   airworthiness   categories.   According   to   the   Australian   Civil   Aviation   Safety   Authority   (CASA),   an   aircraft   airworthiness   category   “…   is   essentially   a   homogeneous   grouping   of   aircraft   types   and   models   of   generally   similar   characteristics,   based   on   the   proposed   or   intended   use   of   the   aircraft,   and   their   operating   limitations.”  (CASA  2000)   As  described  in  civil  aviation  Part  21  regulations  [e.g.,  CASR  1998  Part  21  (CASA  2003)  or  CFR  Title  14  FAR   Part  21  (FAR  2009)],  aircraft  types  may  be  certificated  in  two  classifications:     1. Standard   –   broad   categories   of   aircraft   for   which   detailed   prescriptive   codes   of   requirements   (standards  and  limitations)  are  defined  (i.e.,  normal,  commuter,  transport,  normal  rotorcraft,  and   transport  rotorcraft,  etc.),  or       2. Special  –  for  those  aircraft  that  do  not  meet  the  requirements  of  an  applicable  comprehensive  and   detailed   airworthiness   code   as   required   by   a   standard   category   (CASA   2000).   Special   categories   include:   primary,   intermediate,   restricted,   limited   and   light   sport   aircraft,   etc.,   and   the   designations:  experimental,  gliders,  ultralights,  etc.  (CASA  2000).     For   all   aircraft   certificated   in   a   special   category   of   airworthiness,   operational   limitations   become   part   of   the   conditions   of   the   COA   (FAA   2004b).   These   limitations   may   include   restrictions   on   manoeuvres,   speed,   number   of   passengers,   activities   undertaken   and   where   flights   may   be   conducted.   Experimental   certificates   are   issued   to   individual   aircraft   and   only   for   specific   activities   (e.g.,   research   and   development,   demonstration   and   training,   etc.).   An   aircraft   operating   under   an   experimental   certificate   cannot   be   flown   for   commercial   reward.     Special  flight  permits  may  also  be  granted  to  an  aircraft  that  does  not  meet  the  applicable  airworthiness   requirements,   but   can   reasonably   be   expected   to   be   capable   of   safe   flight   for   a   specified   list   of   activities   (e.g.,   demonstration,  delivery  of  an  aircraft,  search  and  rescue,  or  assisting  in  a  state  of  emergency).    

©  Copyright  2011  Reece  Clothier  

 

Page  5  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

2.2 Current  Airworthiness  Certification  Regulations  for  Civil  UAS     Currently,   there   are   no   specific   standards   and   regulations   for   the   type   certification   of   civil   UAS.   In   their   absence,  the  risks  to  people  and  property  on  the  ground  are  assured  through  substantial  restrictions  on  where   UAS   operations   can   take   place.   An   individual   UAS   may   be   certificated   in   the   experimental   designation   [e.g.,   as   described   in   AC   21-­‐43(0)   (CASA   2006)],   for   specific   applications   and   not   for   commercial   reward,   but   remain   subject   to   operational   restrictions   [e.g.,   in   Australia   these   restrictions   are   contained   in   CASR   Part   101   (CASA   2004)].  Operational  regulations  such  as  CASR  Part  101  (CASA  2004),  prescribe  the  requirement  for  a  COA  based   on   the   nature   of   the   operation   performed   [e.g.,   whether   the   unmanned   aircraft   (UA)   is   operated   over   a   populous  area  or  not].  The  regulations  do  not  define  type  categories  or  categories  of  airworthiness  for  which   COA  can  be  issued.   The   absence   of   an   airworthiness   certification   framework,   and   the   subsequent   operational   limitations   imposed,   come   at   significant   expense   to   the   UAS   industry.   Requisite   to   the   realisation   of   routine   UAS   operations   in   the   NAS   are   regulations   that   facilitate   the   certification   of   an   UAS   as   airworthy.   At   the   highest   level  these  regulations  comprise:     1. a   certification   framework   specifying   the   conditions   for   prescribing   different   airworthiness   regulations   to   different   types   of   UAS   operations,   i.e.,   a   framework   equivalent   in   regulatory   function  to  civil  CPA  regulation  Part  21;  and     2. standards,   procedures   and   recommended   practices   governing   the   design,   manufacture,   maintenance  and  operation  of  UAS  tailored  to  each  of  the  airworthiness  certification  categories,   i.e.,   airworthiness   standards   equivalent   in   regulatory   function   to   the   prescriptive   codes   of   requirements  contained  in  civil  CPA  regulations  Part  23,  25,  27,  29,  etc.     Much   effort   has   been   directed   towards   addressing   the   second   of   these   two   components.   In   particular,   regulations  that  provide  assurances  in  the  airworthiness  of  the  “system”,  which  encompasses  the  UA,  human   elements,   communications   and   ground   infrastructure,   as   opposed   to   that   of   just   an   “aircraft”.   For   example,   Hayhurst   et   al.   (2006)   and   Luxhøj   (2009)   identify   unique   hazards   that   must   be   addressed   in   low-­‐level   airworthiness   regulations   (i.e.,   standards,   and   operational   requirements   covering   unique   aspects   of   UAS   including:   autonomy,   communication   links,   and   ground   control   systems,   etc.).   Standards   development   organisations,   such   as   ASTM   Committee   F38,   and   the   NATO   Standardization   Agency   (NATO   2009),   have   also   made   progress   in   defining   airworthiness   standards   specific   to   UAS.   However,   limited   research   has   been   conducted   into   the   definition   of   the   overarching   certification   framework   that   dictates   the   conditions   for   promulgation  of  these  low-­‐level  regulations.     The  apparent  consensus,  a  default  position  of  regulatory  groups,  is  that  the  airworthiness  framework  for   UAS  should  be  based  on  that  for  CPA  of  the  same  category  (Dalamagkidis  et  al.  2008b),  with  a  number  of  novel   strategies   being   proposed   for   determining   equivalency   between   categories   of   UAS   and   CPA   [e.g.,   see   Grimsley   (2004)].   However,   to   prescribe   the   same   airworthiness   framework   to   UAS   fails   to   address   significant   differences  between  the  two  technologies.  As  described  by  DeGarmo  (2004):     Premising   a   UAV   regulatory   structure   based   on   manned   aircraft   makes   sense,   but   developing   such   regulations   to   cover   the   vast   array   of   UAVs   will   be   a   challenge.   There   are   too   many   differences,   especially   concerning   the   small   UAVs.   Therefore,   expectations   that   all   UAVs   can   conform   to   existing   regulatory   requirement  may  not  be  realistic.   p.  2-­‐46,  DeGarmo  (2004)     Amongst  many  others,  Hayhurst  et  al.  (2006),  DeGarmo  (2004),  Dalamagkidis  et  al.  (2008b),  and  Clothier   et   al.   (2008)   identify   numerous   differences   between   the   risk   paradigms   of   CPA   and   UAS   that   influence   the   development  of  airworthiness  regulations  for  UAS.  The  following  section  describes  some  of  these  differences.   The  objective  is  not  to  categorically  prove  that  the  existing  CPA  airworthiness  certification  framework  would   not  work  for  UAS,  but  rather,  to  justify  the  need  to  explore  potentially  more  effective  strategies.    

2.3 Challenges  of  UAS     Historically  the  primary  purpose  of  airworthiness  regulations  for  civil  CPA  has  been  to  ensure  the  safety  of   the   people   over-­‐flown   (Haddon   and   Whittaker   2002;   JAA   2004);   however,   with   the   rise   of   commercial   ©  Copyright  2011  Reece  Clothier  

 

Page  6  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

passenger  operations,  the  focus  now  includes  aircrew  and  passengers  (JAA  2004).  Analysis  of  accident  statistics   reveals   that   the   primary   people   at   risk   due   to   the   hazards   of   CPA   operations   are   those   onboard   the   aircraft   (Clothier  and  Walker  2006).  Consequently,  CPA  safety  regulations  implicitly  aim  to  limit  or  eliminate  harm  to   those   aboard   the   aircraft,   and   secondarily   to   those   over-­‐flown   (Hayhurst   et   al.   2006).   This   prioritisation   facilitates   the   “common   philosophy”   (Haddon   and   Whittaker   2002)   foundational   to   CPA   airworthiness   regulations,   specifically   the   philosophy   that   “as   far   as   is   practicable,   they   (airworthiness   codes   of   regulatory   requirements)  avoid  any  presumption  of  the  purposes  for  which  the  aircraft  will  be  used  in  service”  (Haddon   and  Whittaker  2002)  and  hence  are  largely  independent  of  the  regions  being  over-­‐flown.  This  is  a  defensible   position   for   CPA,   as   there   will   always   be   someone   at   risk   (i.e.,   at   a   minimum   the   pilot)   and   that   addressing   the   risks   to   those   onboard   an   aircraft   will   inherently   address   the   risks   to   those   over-­‐flown   (Haddon   and   Whittaker   2002;  Clothier  and  Walker  2006).     For  UAS,  it  cannot  be  assumed  that  there  are  people  onboard  the  aircraft.  The  primary  risks  due  to  UAS   operations  are  to  entities  of  value  (EOV)  that  are  external  to  the  system  (i.e.,  external  to  the  immediate  subject   of   regulation).   These   EOV   include   the   people   and   property   over-­‐flown   and   other   airspace   users   within   the   operational   environment.   For   regulatory   matters   pertaining   to   the   integration   of   UAS   operations   within   the   NAS,   the   primary   EOV   are   other   aircraft   and   the   people   onboard   them   through   the   hazard   of   mid-­‐air   collision.   If  the  primary  consequence  of  concern  was  the  degree  of  damage  to  the  aircraft,  then  airworthiness  could  be   considered  largely  independent  of  the  operating  environment  (although  one  could  argue  that  type  of  terrain   and   other   environmental   conditions   would   play   a   part).   However,   recalling   Annex   8   to   the   ICAO   Convention   (ICAO  2005),  the  primary  risks  of  concern  are  to  the  EOV  (i.e.,  the  people  and  property)  over-­‐flown,  which  have   the   potential   to   experience   harm   due   to   falling   parts   or   a   crashing   aircraft.   Thus,   the   levels   of   risk   to   be   controlled   by   UAS   airworthiness   regulations   are   highly   dependent   on   where   the   system   is   operated.   For   example,  a  large  UAS  operated  over  a  built-­‐up  residential  area  presents  a  higher  level  of  risk  (with  respect  to   the   people   and   property   over-­‐flown)   than   the   same   UAS   operated   over   an   unpopulated   area.   Graphical   illustrations  of  the  nature  of  this  dependency  are  provided  in  Weibel  and  Hansman  (2004),  Clothier  and  Walker   (2006),  and  Dalamagkidis  et  al.  (2009).  Thus,  the  required  certification  basis  for  a  particular  UAS  is  a  function  of   the   system   and   the   properties   of   its   intended   operational   environment   (e.g.,   the   density   and   distribution   of   EOV  over-­‐flown,  sheltering,  etc.).  McGeer  et  al.  (1999)  encapsulate  the  nature  of  the  problem:     …  with  a  manned  aircraft  you  have  to  build  to  the  same  standard  no  matter  what  is  underneath  you,  but   among  unmanned  aircraft,  acceptable  safety  for  flights  exclusively  over  oceans  can  be  achieved  with  rather   more   rickety   machines   than   would   be   fit   to   fly   over   a   city.   Hence   the   abundance   of   possibilities   which   everyone  recognises  and  is  struggling  to  manage.     p.  11,  McGeer  and  Vagners  (1999)     To  further  complicate  the  problem,  there  is  significant  diversity  in  UAS  compared  to  CPA.  The  comparative   histograms  of  UA  and  CPA  maximum  takeoff  weight  (MTOW)  and  maximum  operating  speed  shown  in  Figure  1   and   Figure   2,   respectively,   provide   a   graphical   illustration   of   the   magnitude   of   this   diversity.   Evident   from   Figure   1   and   Figure   2,   is   that   the   MTOW   of   the   UAS   fleet   ranges   from   a   few   grams   through   to   hundreds   of   tonnes,  whereas  for  the  CPA  fleet,  the  MTOW  ranges  from  a  few  hundred  kilograms  through  to  thousands  of   tonnes.   The   existing   certification   categories   provided   in   CPA   regulations   do   not   adequately   cover   this  range.   Also  evident  in  the  histograms  are  UAS  types  that  can  operate  in  much  lower  and  higher  speed  regimes  than   that   of   the   CPA   fleet.   Both   speed   and   MTOW   are   factors   that   influence   the   risks   presented   to   people   and   property  on  the  ground.  Hence,  the  risk  profile  to  be  managed  for  UAS  is  larger  than  that  for  the  CPA  fleet.  A   certification   framework   should   ensure   that   risks   are   managed   systematically   across   the   diversity   of   systems   and   operations   (i.e.,   that   low-­‐level   airworthiness   regulations   are   prescribed   commensurate   with   the   degree   of   risk   associated   with   the   operations   for   all   types   of   UAS).   If   the   CPA   airworthiness   certification   framework   is   applied   to   UAS,   it   would   not   be   possible   to   objectively   show   compliance   to   the   ELOS   requirement   at   a   minimum  imposed  cost,  particularly  for  those  types  and  operations  that  are  unique  to  UAS.      

©  Copyright  2011  Reece  Clothier  

 

Page  7  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

 

  6 Figure  1  –  Histogram  of  aircraft  maximum  take-­‐off  weight    

 

 6

Figure  2  –  Histogram  of  aircraft  maximum  operating  speed   6

                                                                                                                                     

 UAS  data  supplied  from  a  database  compiled  and  maintained  by  the  Defence  Science  and  Technology  Organisation  (DSTO),  Australia.   CPA  data  obtained  from  Aviation  Week  and  Space  Report  (http://www.aviationweek.com)  and  Jane's  All  the  World's  Aircraft   (http://catalog.janes.com/catalog/public/index.cfm?fuseaction=home.ProductInfoBrief&product_id=96083).   ©  Copyright  2011  Reece  Clothier  

 

Page  8  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

For  CPA  a  single  certification  category  typically  covers  a  broad  range  of  applications  for  the  particular  type.   Take,   for   example,   a   helicopter   performing   traffic   surveillance   over   highly   populated   areas   and   inspecting   power   lines   in   remote   areas.   With   the   exception   of   specific   payloads,   pilot   training   and   other   operational   approvals   both   operations   could   be   conducted   under   the   same   COA.   Consider   the   same   applications   being   performed   by   an   unmanned   helicopter.   A   higher   level   of   regulation   would   be   required   for   the   unmanned   helicopter  to  be  able  to  perform  traffic  monitoring  over  a  populated  area  than  that  required  to  monitor  power   lines   in   remote   regions.   This   is   due   to   the   dependency   on   the   operational   environment   over-­‐flown.   A   single   UAS  design  may  be  used  for  a  wide  range  of  applications,  from  fire  fighting,  search  and  rescue  over  a  remote   region,  tracking  wildlife  migrations,  to  persistent  surveillance  over  a  heavily  populated  area.   As  illustrated  in   the  above  example  and  as  further  discussed  by  Clothier   et  al.  (2006;  2008),  the  different  UAS  applications  may   be   required   to   meet   different   airworthiness   standards.   There   may   also   be   cases   where   the   level   of   system   airworthiness   could   be   determined   by   the   cost   of   system   attrition,   as   opposed   to   the   required   level   of   reliability  of  an  “equivalent”  CPA  performing  the  same  operation  (McGeer  et  al.  1999;  McGeer  2007;  Clothier   et  al.  2008).  Hence,  the  “common  philosophy”  described  by  Haddon  and  Whittaker  (2002)  that  underpins  the   certification  of  CPA  may  not  be  directly  applicable  to  UAS.   For  CPA,  a  standard  COA  is  issued  against  a  type  certificate   that  prescribes  a  certification  baseline  (a  fixed   description  of  the  aircraft  components  and  their  approved  configuration).  Some  UAS,  particularly  small  UAS,   can  be  highly  modular  in  their  airframes,  payloads  and  supporting  equipment,  allowing  them  to  be  quickly  and   radically   tailored   to   a   particular   mission   or   environment.   For   example:   fuselage   and   wing   sections   can   be   changed   between   operations.   It   may   be   difficult   to   define   a   single   certification   baseline   for   a   UAS   type   and   instead  a  large  number  of  configurations  may  need  to  be  defined  as  part  of  the  type  certification.   Finally,   there   is   also   a   range   of   mitigation   measures   commonly   used   by   UAS,   for   example,   parachute   recovery  systems,  frangibility,  and  self-­‐termination  systems,  all  of  which  may  be  implemented  to  reduce  the   risk   that   a   UAS   presents   to   the   EOV   over-­‐flown.   Within   the   CPA   certification   framework,   such   mitigation   measures   are   evaluated   on   a   case-­‐by-­‐case   basis,   as   they   are   not   traditionally   considered   part   of   the   certification  baseline  for  a  standard  COA.  For  UAS,  these  mitigation  strategies  may  form  a  significant  part  of   the  safety  case;  therefore,  a  systematic  method  for  accommodating  them  is  needed.    

2.3.1 Summary    Among  others,  McGeer  and  Vagners  (1999),  DeGarmo  (2004),  Hayhurst  et  al.  (2006),  Dalamagkidis  et  al.   (2008b),  and  Clothier  et  al.  (2006,  2008)  identify  numerous  differences  between  UAS  and  CPA  that  challenge   the  suitability  of  an  off-­‐the-­‐shelf  approach  for  the  regulation  of  UAS.  The  principal  issue  identified  being  the   difference  in  the  risks  to  be  managed  by  the  airworthiness  regulations.   In   summary,   the   level   of   airworthiness   for   civil   UAS   should   be   determined   by   the   potential   for   harm   to   people  and  property  over-­‐flown,  which  is  a  function  of  the  reliability  of  the  system  and  a  function  of  where  it  is   operated.   On   the   other   hand,   for   CPA   the   level   of   airworthiness   is   principally   defined   by   the   risk   to   those   onboard   and   is   largely   treated   independent   of   the   region   over-­‐flown   (Haddon   and   Whittaker   2002).   As   a   consequence,   the   application   of   the   CPA   airworthiness   certification   framework   (i.e.,   existing   airworthiness   certification   categories   and   low-­‐level   regulations   that   are   mandated   independent   of   the   area   over-­‐flown)   to   civil  UAS  could  result  in:     • the   inequitable   management   of   the   risks   across   the   different   airworthiness   certification   categories  (i.e.,  different  categories  being  managed  to  different  levels  of  risk);     • different   UAS   operations   over   particular   regions   may   be   over-­‐regulated,   or   worse,   present   an   unacceptable   level   of   risk   [refer   to   illustrations   of   geospatial   operational   dependency   provided   in   Weibel  and  Hansman  (2004),  Clothier  and  Walker  (2006),  and  Dalamagkidis  et  al.  (2009)];  or   • the   imposition   of   unjustified   costs   to   the   UAS   industry   [as   shown   in   the   example   presented   by   Clothier  et  al.  (2008)].     Recalling   that   the   primary   purpose   of   air-­‐safety   regulations   is   to   provide   assurances   in   the   safety   of   aviation   at   a   justifiable   cost   to   the   industry;   then   the   justification   for   applying   the   existing   CPA   certification   framework  to  UAS  on  the  basis  of  safety  is  significantly  weakened.  Other  issues  identified  included  unique  UAS   types,  operations  and  risk  profiles,  and  the  dynamic  configuration  of  some  UAS.       To   summarise,   the   CPA   framework   consisting   of   a  small   number   of   type-­‐certification   categories   that   are   mandated   independent   of   the   operation   may   not   provide   the   flexibility   required   to   accommodate   the   spectrum  of  risk  associated  with  the  diversity  of  UAS,  their  operational  environments,  their  applications,  the   reconfigurable  nature  of  the  systems,  and  potential  mitigation  strategies  that  are  readily  employed.  The  direct   application   of   the   existing   CPA   airworthiness   framework   may   not   ensure   the   consistent   and   equitable   ©  Copyright  2011  Reece  Clothier  

 

Page  9  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

management  of  the  risks  associated  with  UAS  operations  and  may  not  justify  the  costs  imposed  on  the  UAS   industry   and   operators.   It   is   important   to   note   that   the   authors   do   not   make   the   categorical   assertion   that   the   application   of   the   existing   CPA   regulatory   framework   will   not   work   for   UAS,   nor   that   existing   low-­‐level   standards  and  regulations  (e.g.,  those  contained  in  Part  25)  are  not  relevant  to  UAS.  Instead,  the  emphasis  is   on  the  identification  of  some  limitations  in  the  structure  governing  their  application  to  UAS;  and  hence  there  is   justification  to  explore  an  alternative  structure  (i.e.,  an  equivalent  Part  21  regulation  specifically  for  UAS).      

3 A  Risk-­‐Management  Approach   One   approach   to   the   regulation   of   airworthiness   for   UAS   is   to   prescribe   standards   and   regulations   on   a   case-­‐by-­‐case  basis  dependent  on  the  degree  of  risk  of  the  operation.  The  approach  has  been  referred  to  as  a   safety-­‐target   approach   (Haddon   and   Whittaker   2002)   and   has   been   advocated   for   the   regulation   of   smaller   UAS  (McGeer  et  al.  1999).  The  safety-­‐target  approach  is  justifiable  and  offers  flexibility  in  the  regulation  of  a   diverse   and   dynamic   industry.   However,   as   discussed   by   Haddon   and   Whittaker   (2002)   a   number   of   disadvantages   arise.   In   particular,   there   are   issues   concerning   the   practicality   of   implementation   and   international  standardisation  and  harmonisation.  In  addition,  the  absence  of  general  prescriptive  codes  of  low-­‐ level   airworthiness   regulations   may   lead   to   subjectivity   as   to   how   applicants   interpret   regulations   and   to   inconsistencies   in   the   evaluation   of   a   safety   case   by   the   regulator.   Consequently,   trade-­‐offs   can   exist   between   the:     1. flexibility  afforded  by  the  framework,   2. consistency  of  the  regulatory  oversight  afforded,  and     3. practicality  of  implementation.          To  ensure  a  workable  management  strategy,  this  paper  proceeds  on  the  basis  that  it  will  be  necessary  to   define   fixed   airworthiness   categories   within   the   diversity   of   UAS   and   their   operations.   However,   it   is   the   authors’   belief   that   some   of   the   defined   airworthiness   categories   (i.e.,   small   UA   and/or   operations   over   unpopulated   regions)   may   be   more   effectively   regulated   under   a   safety-­‐target   approach.   Thus,   in   this   paper   the  method  of  regulation  (i.e.,  whether  a  prescriptive  code  of  regulation  or  safety-­‐target)  to  be  applied  to  each   UAS   airworthiness   category   is   not   prescribed.   This   paper   instead   addresses   the   crux   of   the   problem,   determining   the   framework   of   airworthiness   categories   (e.g.,   what   constitutes   a   small   or   low-­‐risk   UAS?)   to   which  regulations  and  standards  may  then  be  tailored.   The   purpose   of   airworthiness   regulations   is   to   provide   assurances   that   the   risks   associated   with   the   operation   of   UAS   over   populous   areas   are   managed   to   an   appropriate   level.   Secondarily,   the   process   of   defining   these   regulations   should   be   cognisant   of   the   potential   costs   imposed   on   the   industry   and   the   reduction  in  benefit  to  society.  Solving  such  a  multi-­‐objective  problem  is  the  outcome  of  a  risk-­‐management   process.   According   to   internationally   accepted   standards   (AS/NZS   2009),   the   implementation   of   the   risk-­‐ management   framework   encompasses   processes   for   the   identification,   assessment,   evaluation,   mitigation,   communication  and  monitoring  of  a  hazardous  activity  or  technology.  All  components  of  the  risk-­‐management   activity  are  relevant  to  the  development  of  regulations  for  UAS;  however,  of  specific  interest  in  this  paper  are   risk  matrices,  a  tool  widely  used  to  evaluate  risk.     Risk   matrices   provide   a   simple   and   clear   framework   for   the   systematic   review   of   individual   risks   and   portfolios   of   risk   (Cox   2008).   A   risk   matrix   structures   a   quantitative   or   qualitative   assessment   of   risk   into   its   fundamental  components:  a  loss  outcome  for  a  given  scenario  and  a  measure  of  the  uncertainty  in  realising   that   scenario   and   outcome.   A   qualitative,   continuous   or   ranked   scale   is   defined   for   each   component,   and   together  they  form  the  dimensions  of  the  risk  matrix.  The  columns  of  the  risk  matrix  may  be  defined  through   the   discretisation   of   the   range   of   potential   loss   or   the   ordinal   ranking   of   qualitative   specifications   of   loss.   Similarly,  rows  can  be  defined  through  the  discretisation  of  the  range  of  uncertainty  or  the  ordinal  ranking  of   qualitative   specifications   of   uncertainty.   It   is   important   to   preserve   orthogonality   in   the   specification   of   the   two   component   axes,   a   property   fundamental   to   the   concept   of   risk   as   a   multidimensional   and   multi-­‐ echeloned  space  (Kaplan  et  al.  2001;  Clothier  et  al.  2011),  as  opposed  to  the  more  simplistic  notion  of  risk  as  a   single-­‐dimensional   measure.   The   output   rows   and   columns   then   provide   a   contiguous   and   complete   partitioning   of   the   risk   space   with   respect   to   its   two   components   of   loss/harm   and   uncertainty.   These   components  are  briefly  described  in  the  following  sections.  

©  Copyright  2011  Reece  Clothier  

 

Page  10  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

3.1 Loss   Loss  must  be  defined  with  respect  to  an  EOV,  a  property/attribute  of  that  entity,  and  an  associated  scale   describing  the  level  loss  to  the  attribute  (Clothier  et  al.  2011).     The   specification   of   what   constitutes   loss   will   also   depend   on   which   stakeholder   is   performing   the   assessment.   The   “vector   of   loss”   is   the   finite   set   of   types   of   loss   considered,   which   could   include   people,   buildings,  flora  and  fauna,  and  less  tangible  EOV  to  society.  In  defining  the  vector  of  values,  the  comprehensive   management  of  the  scenario  may  necessitate  consideration  of  secondary  hazards  (e.g.,  the  ensuing  collapse  of   a   building,   bushfire,   or   release   of   environmental   contaminants)   that   may   have   an   impact   on   different   EOV.   Under   some   circumstances   the   potential   loss   from   secondary   hazards   could   be   greater   than   that   associated   with  the  occurrence  of  the  primary  hazard.  In  addition,  a  single  hazard  may  have  an  impact  across  a  range  of   EOV-­‐attributes.  For  example,  for  a  large  UAS,  the  primary  attribute  of  concern  is  typically  the  potential  physical   harm  to  a  person  or  building.  For  a  small  UAS,  striking  a  house  may  have  a  greater  psychological  impact  on  the   occupant  than  that  of  a  physical  impact  on  the  occupant  or  the  building  itself.  The  appropriate  management  of   the   UAS   fleet   may   therefore   need   to   consider   a   range   of   EOV   and   EOV-­‐attributes   for   which   loss   may   be   registered  (e.g.,  for  people  this  could  include  physical,  psychological,  or  financial  attributes).       Associated   with   each   attribute   is   a   continuous   or   discrete   spectrum   describing   the   degree   of   loss   (Clothier   et  al.  2011).  This  spectrum  may  be  expressed  through  quantitative  or  qualitative  measures.  A  type  of  loss  can   be   defined   independent   of   a   particular   hazard   (e.g.,   for   physical   harm   to   people   this   could   be   no   injury,   minor   injury,  serious  injury  or  fatality).  The  spectrum  of  loss  should  consider  the  potential  loss  to  an  individual  EOV  as   well   that   to   applicable   EOV   groups   (e.g.,   for   physical   injury   to   people,   a   spectrum   of   loss   could   be   defined   ranging  from  no  injury  to  a  single  person  through  to  multiple  fatalities  within  a  group  of  people).   Society  may   also   place   a   higher   value   on   certain   sub-­‐types   of   EOV   (e.g.,   the   distinction   between   first-­‐   and   third-­‐party   people   and   property).   Society   subsequently   may   assign   a   higher   level   of   loss   to   these   EOV,   despite   the   measure  of  the  loss  outcome  (e.g.,  a  fatality)  being  the  same.  

3.2 Uncertainty    

Klinke  and  Renn (2002)  state  that  there  is  no  established  classification  of  uncertainty  and  that  it  is  a  topic   of   major   debate   within   the   risk   community.   For   the   purposes   of   this   paper,   the   high-­‐level   concept   of   uncertainty  is  the  state,  even  partial,  of  deficiency  of  information  related  to  the  understanding  or  knowledge   of  a  loss  outcome,  inclusive  of  all  of  its  components  and  its  component-­‐relationships  [definition  modified  from   ISO  Guide  73  (2009)].  This  definition  encompasses  uncertainty  that  arises  through  stochastic  variation  or  a  lack   of  knowledge  of  the  scenario  leading  to  the  loss  outcome,  the  particular  level  of  loss,  and  the  likelihood  of  its   occurrence.   In  a  traditional  risk  matrix,  the  concept  of  uncertainty  is  narrowed  to  a  measure  of  the  potential  of  a  loss   event   occurring.   A   range   of   measures   can   be   used   to   describe   potential,   including   probability   (DoD   2000),   likelihood   (ISO   2009),   frequency   (Cox   2008),   and   expected   value.   These   measures   can   be   expressed   on   qualitative   or   quantitative   scales.   For   example,   MIL-­‐STD-­‐882D   defines   five   probability   levels:   improbable,   remote,   occasional,   probable,   and   frequent   [refer   to   Table   A-­‐II   of   DoD   (2000)].   To   generalise,   and   without   prescribing   the   particular   method   of   assessment,   this   dimension   can   be   thought   of   as   a   scale   describing   the   potential  for  realising  loss.  

3.3 Risk   The  cells  of  the  matrix  represent  an  assessment  of  the  risk  for  a  given  scenario.  Although  widely   used,  we   argue  that  the  quantitative  assessment  of  risk  is  not  the  arithmetic  multiplication  of  loss  and  uncertainty.  An   assessment  of  risk  is  the  Cartesian  product  of  a  level  of  loss  (i.e.,  a  given  column)  by  a  degree  of  uncertainty   (i.e.,   a   given   row),   and   the   output   is   the   set   of   all   ordered   pairs   of   the   elements   from   the   two   sets   (i.e.,   the   ordered   set   of   the   pair-­‐wise   combination   of   the   elements   of   the   components   of   risk:   those   of   loss   and   uncertainty).  Thus,  an  assessment  of  risk  may  be  thought  of  as  a  mapping  within  a  multidimensional  space  (the   generation  of  the  cells  of  the  matrix).     A   particular   assessment   of   the   risk   may   be   compared   against   risk   criteria   or   ranked   alongside   assessments   made   for   other   scenarios.   Systematic   decisions   may   then   be   made   as   to   the   setting   of   controls   and   the   appropriate   treatment   of   the   risks   to   ensure   criteria   are   satisfied   and   that   risks   are   managed   equitably   over   the  industry  (i.e.,  across  all  scenarios).     To   summarise   in   the   context   of   developing   an   airworthiness   certification   framework   for   UAS,   a   risk   matrix   provides   a   simple   and   accepted   tool   for   the   systematic   assessment,   comparison   and   ranking   of   risks   (e.g.,   ©  Copyright  2011  Reece  Clothier  

 

Page  11  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

those  risks  associated  with  the  operation  of  different  types  of  UAS  over  inhabited  areas).  This  information  may   then   be   used   to   guide   the   setting   of   risk   controls   (i.e.,   the   tailoring   of   airworthiness   regulations)   to   ensure   risk   criteria   (e.g.,   the   ELOS   objective)   are   met.   Thus,   it   is   the   hypothesis   of   this   paper   that   a   risk   matrix   could   provide  the  guiding  principles  necessary  to  structure  an  airworthiness  certification  framework  for  civil  UAS.    

3.4 Application  to  the  Development  of  Part  21  Regulations  for  UAS   A  risk  matrix  is  used  as  inspiration  for  the  structuring  of  an  airworthiness  certification  framework  and  the   resultant   airworthiness   certification   matrix   (ACM)   is   illustrated   in   Figure   3.   The   components   (axes,   cells   and   cell-­‐values)  of  the  ACM  are:     1. Type  category  of  UAS  (the  columns)  –  This  dimension  of  the  ACM  is  defined  based  on  a  discrete,   continuous   and   increasing   scale   of   loss   (or   consequence).   Each   type   category   thus   represents   a   grouping   of   UAS   where,   given   the   occurrence   of   an   unrecoverable,   flight-­‐critical   failure   and   independent  of  any  particular  area  over-­‐flown,  the  potential  magnitude  of  the  resultant  damage   that   could   be   caused   to   EOV   over-­‐flown   is   of   a   similar   magnitude.   An   example   of   such   a   categorisation  is  provided  by  Clothier  et  al.  (2010);     2. Category  of  operational  environment  (the  rows)  –  This  dimension  of  the  ACM  is  defined  based  on  a   discrete,   continuous   ranking   of   potential   for   realising   a   degree   of   loss.   Each   category   of   operational  environment  thus  represents  a  grouping  of  operational  areas  where  the  potential  for   realising  loss,  given  a  UAS  impacting  the  area,  is  of  a  similar  magnitude;     3. Operational  scenarios  (the  cells)  –  This  component  of  the  ACM  is  analogous  to  the  assessment  of   risk  within  a  risk  matrix,  which  is  determined  by  the  Cartesian  product  formed  over  the  sets  of  row   and   column   elements.   The   Cartesian   product   is   used   to   construct   a   matrix   with   a   finite   number   of   cells  (q),  determined  by  the  number  of  rows  (m)  multiplied  by  the  number  of  columns  (n).  Thus,   each  cell  represents  a  unique  operational  scenario  defined  by  the  combination  of  a  specific  UAS-­‐ type  category  together  with  a  specific  category  of  the  operational  environment;  and     4. Airworthiness   certification   categories   (the   values   assigned   to   the   cells)   –   This   component   is   determined  first  by  an  assessment  of  the  risks  associated  with  a  given  operational  scenario  (i.e.,   the   given   cell)   and,   then,   an   assignment   of   the   operational   scenario   to   one   of   r   certification   categories,  based  on  this  assessment,  where  1  ≤  r  ≤  q.     In  general,  and  by  virtue  of  the  definition  of  the  values  assigned  to  the  axes,  the  level  of  risk  associated   with   each   scenario   (and   subsequently   the   certification   categories   assigned   to   them)   defined   in   the   lower-­‐right   quadrant   of   the   ACM   is   higher   than   the   levels   of   risk   associated   with   those   scenarios/categories   situated   towards  the  upper-­‐left  quadrant.   In   order   to   map   the   type   of   UAS   and   operational   area   to   the   spectra   of   increasing   loss   and   potential,   respectively,  it  is  necessary  to  define  a  common  set  of  hazards  and  EOV.  As  will  be  described  in  the  following   sections,   the   condition   for   independence   between   the   rows   and   columns   can   be   maintained   through   conditional  modelling.   In  summary,  the  ACM  provides  a  systematic  method  for  partitioning  the  numerous  possible  combinations   of   UAS   types   and   operations   into   a   finite   number   of   scenarios.   Certification   categories   are   then   assigned   to   each  scenario  (cell)  based  on  the  assessed  levels  of  risk.    

©  Copyright  2011  Reece  Clothier  

 

Page  12  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

Figure  3  –  Illustration  of  an  airworthiness  certification  matrix  for  civil  UAS  (defined  for  a  given  type  of   loss  outcome  and  a  given  stakeholder  perspective)  

 

 

3.4.1 UAS-­‐Type  Categories  (The  Columns)   As   illustrated   in   Figure   3,   each   of   the   n   columns   of   the   ACM   represents   a   finite   UAS-­‐type   category.   In   a   traditional   risk   matrix,   this   dimension   represents   a   scale   of   increasing   loss,   consequence   or   harm   resulting   from  a  mishap.  For  example,  Table  A-­‐I  in  MIL-­‐STD-­‐882D  (DoD  2000)  defines  four  levels  of  severity:  negligible,   marginal,   critical,   catastrophic.   The   basis   for   defining   UAS-­‐type   categories   is   a   spectrum   describing   the   potential  magnitude  of  loss  that  the  different  types  of  UAS  may  cause.  This  spectrum  is  one  of  conditional  and   hypothetical   loss   that   is   independent   of   the   characteristics   of   a   particular   operational   area   (i.e.,   preserving   the   orthogonality  of  the  two  component  axes  of  the  matrix).  More  simply  put:       Given   the   occurrence   of   an   unrecoverable   flight-­‐critical   failure,   what   is   the   maximum   degree   of   loss   the   UAS  could  cause,  irrespective  of  where  it  crashed?       Each  type  category  subsequently  describes  a  grouping  of  UAS  where  the  magnitude  of  potential  loss  due   to  a  mishap  is  within  some  pre-­‐defined  bounds,  irrespective  of  where  the  UAS  is  operated.  The  type  categories   must   be   disjoint   (i.e.,   provide   an   unambiguous   classification   of   UAS)   and   together   provide   complete   and   continuous  coverage  of  the  range  of  plausible  loss  (i.e.,  cover  the  spectrum  of  UAS  types  and  their  ability  to   cause  loss).  The  fundamental  concept  of  loss  as  a  basis  for  categorisation  may  be  observed  in  current  CPA  type   categories  (i.e.,  the  classification  criterion  of  the  number  of  people  onboard);  however,  this  basis  is  not  applied   consistently   across   the   CPA   categorisation   scheme.   Thus,   pivotal   to   the   specification   of   type   categories   for   use   within   the   proposed   ACM   is   an   understanding   of   the   potential   types   and   levels   of   loss   associated   with   UAS   operations.   As  described  in  §3.1,  it  may  be  appropriate  to  consider  more  than  just  the  potential  loss  to  people.  With   respect  to  UAS  operations  over  inhabited  areas,  the  spectrum  should  cover  the  range  of  potential  loss  to  an   individual   EOV   as   well   that   to   applicable   EOV   groups   (e.g.,   for   physical   injury   to   people,   a   spectrum   of   loss   could   be   defined   ranging   from   no   injury   to   a   single   person   through   to   multiple   fatalities   within   a   group   of   people).  

©  Copyright  2011  Reece  Clothier  

 

Page  13  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

3.4.1.1 Categorisation  of  Loss   Proceeding  on  the  premise  that  a  workable  airworthiness  certification  framework  requires  some  degree  of   categorisation  of  the  diversity  of  UAS  types  to  be  operated,  then  a  mechanism  for  discretising  the  spectra  of   loss  associated  with  this  diversity  is  needed.     Predominantly,   the   process   used   in   the   specification   of   existing   categorisation   schemes   has   been   subjective,   providing   little   or   no   objective   justification   for   the   particular   partitioning   chosen.   Often   these   schemes  reflect  the  needs  (and  commercial  desires)  of  the  particular  stakeholders  involved.  Dalamagkidis  et  al.   (2009)   subjectively   assign   categories   based   on   the   “natural   classification”   observed   between   MTOW   and   a   derived  measure  of  reliability.  Given  the  potentially  significant  influence  type  categories  will  have  in  shaping   the   future   of   the   UAS   industry,   a   transparent   and   defensible   strategy   for   determining   a   suitable   number   of   categories   and   the   classification   criteria   that   define   each   category   is   needed.   One   such   method,   concerned   with  the  risk  of  human  casualties  is  discussed  by  Clothier  et  al.  (2010).   Typically,  categorisation  processes  seek  to  collapse  the  distinct  spectra  of  loss  outcomes  (as  described  in   the   previous   section)   into   a   single   loss   dimension   for   which   discrete   categories   are   then   defined.   For   example,   the   loss   dimension   defined   in   MIL-­‐STD-­‐882D   (DoD   2000)   includes   the   distinct   loss   outcomes   to   people,   property  and  the  environment.  The  combination  of  these  outcomes  onto  a  single  dimension  does  not  preserve   the   distinct   likelihood   of   realising   each   individual   loss   outcome.   In   addition,   such   processes   represent   a   subjective  judgement  on  the  comparative  value  that  stakeholders  place  on  the  different  EOV  [e.g.,  in  MIL-­‐STD-­‐ 882D   (DoD   2000)   the   mishap   severity   category   of   catastrophic   equates   the   loss   of   a   human   life   to   that   of   permanent  total  disability,  to  irreversible  environmental  damage,  and  to  the  loss  of  more  than  US$1  million].     In   contrast,   the   approach   proposed   in   this   paper   seeks   to   preserve   the   separate   dimensions   of   loss   and   acknowledge  subjectivity  in  the  specification  of  loss.  The  matrix  illustrated  in  Figure  3  is  defined  for  a  particular   stakeholder   perspective   and   type   of   loss   outcome;   however,   the   dimension   of   loss   may   also   be   defined   by   the   distinct   spectra   of   loss   outcomes   specified   by   each   individual   stakeholder   involved   in   the   decision-­‐making   process.  The  result,  illustrated  in  Figure  4,  is  a  set  of  matrices  describing  the  loss  outcomes  of  concern  for  each   particular  stakeholder.      

Figure  4  –  Preservation  of  the  different  types  of  loss    

©  Copyright  2011  Reece  Clothier  

 

 

Page  14  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

3.4.1.2 Summary   Currently,   there   is   no   consensus   on   the   definition   of   type   categories   for   UAS   (DeGarmo   2004;   Dalamagkidis  et  al.  2008b).  Many  of  the  existing  schemes  have  not  been  defined  specifically  for  the  purposes   of   certification.   For   a   comprehensive   review   of   UAS   categorisation   schemes   refer   to   Nas   (2011).   In   addition,   there  is  no  consensus  on  a  process  for  defining  a  suitable  scheme  (DeGarmo  2004).  As  described  by  DeGarmo   (2004),   consensus   on   a   particular   UAS-­‐type-­‐categorisation   scheme   is   fundamental   to   the   progress   of   regulations.     Under   the   proposed   ACM,   a   type-­‐categorisation   scheme   for   UAS   should   be   defined   based   on   the   qualitative  or  quantitative  specification  of  the  potential  levels  of  loss  to  people,  property,  and  other  EOV  over-­‐ flown.   To   preserve   the   generality   of   the   proposed   ACM   framework,   the   vector   of   values   has   not   been   prescribed.   Instead   (and   as   discussed   in   Sections   §3.1,   §3.4.1   and   §3.4.1.1)   the   specification   of   type   categories   for   UAS   may   need   to   consider   a   range   of   loss   outcomes   in   order   to   appropriately   manage   the   diverse   risk   profile  associated  with  UAS  and  their  operations.   Finally,   there   will   be   factors   that   may   not   be   directly   related   to   loss   that   need   to   be   taken   into   consideration.  For  example,  it  may  be  practical  to  distinguish  between  rotorcraft  UAS  and  fixed-­‐wing  UAS  (as  is   done  in  the  airworthiness  framework  for  CPA)  due  to  the  unique  aspects  of  each  type  and  the  availability  of   existing  low-­‐level  codes  of  regulation  that  could  be  applied.  Such  factors  are  related  to  the  context  of  practical   regulation  and  should  be  second-­‐order  considerations  in  the  refinement  of  the  type-­‐categorisation  scheme.     The   above   sections   briefly   introduce   some   of   the   issues   that   need   to   be   considered   in   the   definition   of   type  categories  for  UAS.  A  suitable  process  for  defining  these  categories  is  described  by  Clothier  et  al.  (2010).  

3.4.2 Operational  Categories  (The  Rows)   As   illustrated   in   Figure   3,   each   row   of   the   ACM   represents   one   of   m   categories   of   operational   environment.  The  definition  of  the  operational  categories  can  be  considered  as  representative  of  a  spectrum   of   increasing   potential   for   loss   that   is   determined   independent   of   the   characteristics   of   any   particular   type   category  of  UAS.  More  simply  put:     Irrespective  of  the  type  of  UAS  that  comes  to  earth,  how  susceptible  is  an  area  to  experiencing  loss?     Like   the   specification   of   type   categories,   the   categories   of   operational   environments   should   be   disjoint   (i.e.,   provide   an   unambiguous   classification   of   the   different   types   of   areas   over-­‐flown)   and   together   provide   complete  and  continuous  coverage  of  the  range  of  possible  areas  over-­‐flown  (i.e.,  cover  the  entire  spectrum  of   operational   areas).   It   is   important   to   note   that   independence   between   the   specification   of   operational   and   type  categories  (the  two  component-­‐dimensions  of  the  ACM)  is  maintained,  as  the  assessment  is  for  all  types   of   loss   not   specific   to   a   particular   type   category   of   UAS.   In   essence,   the   operational   categories   are   defined   based  on  the  “susceptibility”  of  the  area  to  loss  given  a  UAS  mishap.  

3.4.2.1 Categorisation  of  Operational  Environments   Under   the   ACM   approach,   the   different   operational   environments   need   to   be   mapped   to   a   scale   of   increasing  potential  for  harm  independent  of  the  particular  type  of  UAS  flying  overhead.  The  characterisation   of   this   axis   of   the   ACM   is   a   complex   problem   requiring   knowledge   of   the   susceptibility,   number   and   distribution  of  different  types  of  EOV  for  which  loss  could  be  registered.  Once  mapped  to  a  common  scale  of   potential,  the  operational  environments  can  be  grouped  into  a  finite  number  of  categories.   The   primary   EOV   of   concern   with   respect   to   safety   is   people,   and   this   provides   the   basis   for   existing   qualitative  categorisation  schemes  identified  in  aviation  literature.  CASA  describes  a  populous  area  as  an  area   of:     …sufficient   density   of   population   for   some   aspect   of   the   operation,   or   some   event   that   might   happen   during  the  operation  (in  particular,  a  fault  in,  or  failure  of,  the  aircraft  or  rocket)  to  pose  an  unreasonable  risk   to  the  life,  safety  or  property  of  somebody  who  is  in  the  area  but  is  not  connected  with  the  operation.   CASR  Part  101.025  (CASA  2004)     Federal  Aviation  Regulations  (FAR  2009)  and  supporting  material  yield  numerous  terms  for  the  description   of  operational  areas:  densely  populated,  congested,  other  than  congested,  unpopulated,  sparsely  populated,   and  over  water.  However,  no  prescriptive  definitions  are  provided.  The  RTCA  Special  Committee  203  (SC-­‐203)   ©  Copyright  2011  Reece  Clothier  

 

Page  15  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

for   Unmanned   Systems   defines   four   qualitative   categories   for   the   classification   of   operational   areas,   specifically:     1. Densely   populated   –   described   as   urban   and   suburban   areas,   particularly   with   residential   developments.     2. Sparsely   populated   –   generally   rural   areas   and   agricultural   regions   where   individual   homes   are   separated  by  large  tracts  of  land.   3. Unpopulated  –  uninhabited  areas.   4. Open-­‐air  assemblies  of  people  –  described  as  outdoor  gatherings  without  overhead  shelter.   DO-­‐304,  p.I-­‐18,  RTCA  (2007)     Prevalent   factors   in   the   above   definitions   are   the   distribution   of   the   population   within   the   area   and   the   degree  of  sheltering  provided.  Thus  the  simplest  specification  of  operational  areas  could  be:     1. Populated  –  given  a  UAS  impacting  the  area,  the  potential  for  loss  is  greater  than  zero.  E.g.,  this   would  be  any  area  where  an  EOV  is  present.     2. Not  populated  –  given  a  UAS  impacting  the  area,  the  potential  for  loss  is  approaching  zero.   E.g.,  a   sanitised  test  range,  a  desert,  or  the  high  seas.     The  alternative  approach  is  to  not  explicitly  classify  operational  areas  into  a  predefined  set  of  categories.   As   illustrated   by   Weibel   and   Hansman   (2004),   Clothier   et   al.   (2007),   and   Dalamagkidis   et   al.   (2009),   it   is   possible   to   use   population   and   other   geographical   information   system   (GIS)   databases   to   gain   a   continuous   quantitative   and   geo-­‐referenced   characterisation   of   the   distribution   of   an   EOV   for   every   operational   area   considered.   The   definition   and   number   of   operational   areas   is   instead   determined   by   the   resolution   of   the   data  available  as  opposed  to  a  finite  number  of  subjectively  defined  categories.     Defining   a   greater   number   of   categories   provides   greater   resolution   of   the   diverse   areas   available   for   UAS   operations.  However,  there  are  disadvantages  in  having  a  large  number  of  operational  categories.  Under  the   proposed   framework,   UAS   issued   with   a   COA   in   a   given   airworthiness   category   cannot   be   flown   over   areas   where   a   higher   category   of   airworthiness   certification   is   required.   As   a   consequence,   “pseudo”   airways   or   pockets  of  UAS  activity  may  eventuate  (i.e.,  similar  to  how  concentrations  of  VFR  air  traffic  can  develop  due  to   the   distribution   of   controlled   airspace).   The   greater   the   number   of   operational   categories   defined,   the   narrower  and  the  denser  these  airways  and  pockets  of  UAS  operations  can  become.  In  addition,  the  greater   the  number  of  categories  defined,  the  more  sensitive  the  qualitative  schemes  are  to  diverse  interpretation  and   hence   the   greater   the   need   for   a   quantified   categorisation   scheme.   A   greater   number   of   operational   categories  can  also  permit  UAS  operations  over  areas  where  previously  they  were  prohibited  from  flying  (i.e.,   through  the  specification  of  sub-­‐categories  within  the  existing  category  of  “populous  area”).  Therefore,  careful   consideration  should  be  given  to  the  specification  of  the  operational  categories,  as  they  can  influence  how  the   airspace  system  is  used  by  UAS.     Finally,  a  practical  implementation  of  the  airworthiness  framework  requires  the  operational  categories  to   be  identified  in  navigation  charts  (e.g.,  similar  to  the  depiction  of  built-­‐up  areas  in  CPA  navigation  charts).  For   UAS,   it   may   be   appropriate   to   develop   separate   charts   representing   the   category   of   operational   area   assigned   to   geographic   regions.   The   more   categories   there   are,   the   more   complicated   such   charts   would   become.   Hence,  the  challenge  is  to  specify  the  optimal  number  of  operational  categories  that  adequately  encapsulates   the  diversity  of  operational  areas  and,  in  turn,  provides  a  workable  regulatory  outcome.   Finally,  there  a  number  of  other  factors  influencing  the  potential  for  causing  harm  to  EOV  and  hence  the   categorisation  of  operational  environments.  These  include,  for  example,  factors  influencing  the  response  of  an   individual  EOV  to  a  mishap  (e.g.,  the  demographic  of  the  population:  age  and  health,   etc.),  the  tendency  for   clustering  within  a  population  of  EOV,  which  is  needed  to  characterise  the  likelihood  of  a  mishap  resulting  in   multiple   causalities,   and   sheltering   [e.g.,   the   population-­‐sheltering   models   used   in   the   Columbia   accident   investigation   (Lin   et   al.   2003)].   The   susceptibility   and   hence   potential   for   an   operational   environment   to   experience   loss   could   also   change   with   the   time   of   day   (e.g.,   due   to   changes   in   the   number   and   distribution   of   people  exposed).  

©  Copyright  2011  Reece  Clothier  

 

Page  16  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

3.4.2.2 Summary   Under   the   proposed   framework   the   categorisation   of   operational   environments   should   be   based   on   the   potential   for   the   realisation   of   loss   given   a   UAS   impacting   that   area.   As   described   above,   a   range   of   factors   characterise   the   conditional   potential   for   loss   in   a   given   environment.   A   method   for   categorising   these   operational   environments   is   also   needed.   The   specification   of   these   categories   requires   the   consideration   of   broader   issues   relating   their   potential   impact   on   the   operation   of   UAS   and   other   airspace   users   and   the   practical  issues  associated  with  the  promulgation  and  management  of  regulations.  

3.5 The  Operational  Scenarios  (The  Cells)     Each   cell   of   the   ACM   describes   a   unique   risk   scenario   defined   by   the   combination   of   a   specific   category   of   operational   area   (a   row)   with   a   specific   UAS-­‐type   category   (a   column).   The   complete   set   of   q   operational   scenarios  is  defined  by  the  Cartesian  product  of  the  two  component  axes  of  loss  and  uncertainty.     The  matrix-­‐based  approach  reduces  the  continuous  spectrum  of  scenarios  to  a  finite  number  of  discrete,   contiguous   and   organised   categories.   The   structure   of   the   matrix   provides   a   simple   mechanism   for   systematically  relating  the  operational  scenarios  to  the  fundamental  components  of  civil  aviation  regulations   (i.e.,   regulations   pertaining   to   the   design,   maintenance   and   manufacture   of   aircraft,   and   the   regulations   pertaining  to  the  operation  of  an  aircraft).   The   next   step   in   the   process   is   to   assign   each   operational   scenario   to   an   airworthiness   certification   category.   Diagrammatically,   and   referring   to   Figure   3,   this   is   the   process   of   determining   the   colour   of   each   cell   within  the  matrix.  

3.6 The  Airworthiness  Certification  Categories  (The  Cell-­‐Values)   Risk  is  often  expressed  as  the  Cartesian  product  of  its  components  of  loss  and  the  potential  for  realising   each  specified  loss.  Each  cell  may  be  characterised  by:     1. a  set  of  loss  outcomes  (scenarios),  and     2. an  associated  set  of  measures  describing  the  uncertainty  in  the  realisation  of  these  loss  outcomes.       These  assessments  may  be  quantitative  or  qualitative  and  may  be  made  for  each  combination  of  UAS-­‐type   and   operational   category   (i.e.,   for   each   operational   scenario   defined   by   the   cells   of   the   matrix).   Typically,   a   single  dimension  is  used  to  describe  the  loss  and  uncertainty  components  of  the  measure  of  risk.  For  example,   Table   A-­‐IV   in   MIL-­‐STD-­‐882D   (DoD   2000)   defines   a   finite   qualitative   scale   for   assessing   risk:   low,   medium,   serious,   and   high.   Quantified   measures   (e.g.,   the   expected   frequency)   are   also   used   to   reduce   the   multi-­‐ dimensional  measurement  problem  to  that  of  a  single  measure.  A  common  example  is  the  expected  number  of   casualties   per   flight   hour,   as   used   in   Grimsley   (2004),   Weibel   and   Hansman   (2004),   and   Clothier   and   Walker   (2006).   However,   as   described   by   Paté-­‐Cornell   (1994)   and   the   Range   Commanders   Council   (RCC,   2007),   the   effective  management  of  the  risks  should  include  measures  of  the:     1. individual  risk  (IR)  to  ensure  that  no  individual  entity  is  disproportionately  exposed  (Jongejan  et  al.   2006);   2. collective   risk   (CR)   to   ensure   the   levels   of   risk,   when   aggregated   across   the   entire   population   of   entities,  are  managed  to  an  acceptable  level;  and     3. societal  risk  (SR)  (Jongejan  et  al.  2006),  also  referred  to  as  catastrophic  risk  (RCC  2007),  to  reflect   society’s  adversity  to  mishaps  that  result  in  a  larger  magnitude  of  consequence.       Each  operational  scenario  then  needs  to  be  assigned  to  a  certification  category.  As  discussed  previously,   this  assignment  should  be  based  on  the  levels  of  risk  determined  for  each  operational  scenario.  This  ensures   that   operational   scenarios   that   present   comparable   levels   of   risk   are   required   to   demonstrate   a   comparable   level  of  airworthiness  (i.e.,  are  subject  to  the  same  body  of  airworthiness  regulations  and  standards).   The  first  step  in  this  assignment  process  is  to  determine  the  number  of  certification  categories  required,   r.   The   minimum   number   of   airworthiness   categories   is   one.   This   is   the   case   where   the   same   category   of   airworthiness   is   assigned   to   all   operational   scenarios   and   represents   the   undesirable   “one-­‐size-­‐fits-­‐all”   regulatory  approach.  In  effect,  this  approach  prescribes  the  same  level  of  airworthiness  regulation  irrespective   of   the   degree   of   loss   a   UAS   is   capable   of   causing   and   irrespective   of   whether   EOV   are   exposed   or   not.   The   maximum   number   of   airworthiness   categories   is   determined   by   the   dimensions   of   the   airworthiness   matrix   ©  Copyright  2011  Reece  Clothier  

 

Page  17  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

(i.e.,  q).  The  more  airworthiness  categories  defined,  the  more  tailored  and  flexible  the  airworthiness  regulatory   framework  is  in  its  management  of  the  diversity  of  UAS  and  their  operations.  Standards  and  regulations  must   be   defined   for   each   airworthiness   category;   hence,   the   greater   the   number   of   categories,   the   greater   the   regulatory  development  task  and  the  more  difficult  it  becomes  to  manage.     To   summarise,   there   is   no   ideal   number   of   categories;   it   will   depend   on   a   number   of   trade-­‐offs   as   identified   above.   In   addition,   there   are   many   subjective   issues   specific   to   the   context   of   the   promulgation   of   a   practical   and   workable   regulatory   framework   that   will   influence   the   setting   of   the   number   of   airworthiness   categories.  A  process  is  then  needed  to  assign  each  operational  scenario  to  the  airworthiness  categories.    

3.6.1 Assigning  Airworthiness  Categories     The  basic  principle  for  assigning  airworthiness  categories  is  to  determine  which  cells  of  the  airworthiness   matrix  pose  a  comparable  level  of  risk  and  to  then  group  these  into  a  single  airworthiness  category  so  that  a   consistent  level  of  regulation  may  be  applied.     A   risk   matrix   [e.g.,   Table   A-­‐III   of   MIL-­‐STD-­‐882D   (DoD   2000)]   provides   one   method   for   mapping   an   operational  scenario  with  a  given  “mishap  severity”  (Table  A-­‐I)  and  a  given  “mishap  probability”  (Table  A-­‐I)  to  a   “risk-­‐assessment   value”   (A-­‐IV).   For   example:   a   mishap   of   catastrophic   severity   that   is   probable   is   assessed   as   a   high   risk   (DoD   2000).   Each   operational   scenario   within   the   ACM   with   a   comparable   risk   assessment   (e.g.,   all   those  cells  assigned  as  being  low  risk)  could  then  be  assigned  to  the  same  certification  category  (e.g.,  Cert  1),   and   so   forth   for   all   other   levels   defined   by   the   qualitative   scale   describing   the   risk-­‐assessment   value   (DoD   2000).   For   quantitative   risk   assessments   presented   on   a   continuous   scale,   one   approach   is   to   try   to   observe   natural  breakpoints  within  the  assessments  of  risk  and  to  use  these  breakpoints  to  delineate  the  airworthiness   categories  (e.g.,  mathematical  clustering  approaches).  The  process  is  complete  when  all  operational  scenarios   (i.e.,  cells)  within  the  matrix  have  been  assigned  to  one  of  the  r  airworthiness  categories.   Given   the   mapping   of   each   cell   of   the   matrix   to   a   particular   airworthiness   category,   the   final   step   is   to   tailor  regulations  to  each  category.  

3.7 Tailoring  of  Regulations     The  primary  purpose  of  safety  regulations  is  to  mandate  controls  that  ensure  that  risk  is  managed  to  an   acceptable   level.   Under   the   ELOS   objective,   the   tailoring   of   UAS   airworthiness   regulations   is   the   process   of   specifying   standards   particular   to   each   airworthiness   category   that   provide   assurance   that   levels   of   risk   are,   at   a  minimum,  equivalent  to  those  of  CPA  operations.  Thus,  assessments  of  the  de  facto  risk  levels  for  CPA  should   be   used   to   define   the   de   manifestis   risk   criteria   for   UAS   (i.e.,   the   existing   CPA   risk   levels   define   the   limit   on   what  is  considered  tolerable  for  UAS).  As  discussed  in  §3.6,  the  UAS  risk  criteria  should  be  specified  in  terms  of   measures   of   the   IR,   CR   and   SR   for   each   loss   outcome   of   concern   (i.e.,   human   casualty,   damage   to   buildings,   etc.).   The   primary   focus   of   airworthiness   regulations   is   to   control   the   potential   occurrence   of   flight-­‐critical   failures.  In  this  paper,  a  failure  is  used  to  describe  any  state  within  the  UAS-­‐airworthiness  system  (e.g.,  people,   processes,   and   equipment   that   are   components   of   the   design,   manufacture,   maintenance,   and   operation   of   the   UAS)   that,   when   realised,   can   lead   to   the   primary   hazard   of   a   UAS   impacting   the   ground   during   flight.   Examples   include   a   failure   of   the   navigation   system,   a   latent   error   in   flight-­‐control   software,   incorrect   maintenance  procedures  leading  to  a  structural  or  propulsion  failure  in  flight,  or  incorrect  operator  commands   that  lead  to  a  controlled  flight  into  terrain.  All  UAS  assigned  to  the  same  airworthiness  certification  category   are  ultimately  subject  to  the  same  body  of  airworthiness  regulations  and  hence  should  exhibit  a  comparable   rate   of   flight-­‐critical   failure.   The   tailoring   of   regulations   is   the   process   of   ensuring   the   potential   for   a   flight-­‐ critical  failure  for  a  given  certification  category,  X,  satisfies  the  following  relationships:     IRCPA    ≥  IRX     CRCPA    ≥  CRX   Equation  1   SRCPA(i)    ≥  SRX(i)     ∀  i  ≥  0,     where   IRCPA,   CRCPA,   and   SRCPA   are   the   measures   of   individual,   collective   and   societal   risk   based   on   the   safety   performance  of  CPA,  respectively.  IRX,  CRX,   and  SRX  are  the  measures  of  individual,  collective  and  societal  risk   determined  for  the  given  airworthiness  category  X,  respectively,  and  i  is  the  plausible  domain  of  a  spectrum  of   loss  outcomes.    

©  Copyright  2011  Reece  Clothier  

 

Page  18  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

The  particular  method  for  solving  the  relationships  given  in  Equation  1  depends  on  the  measure  used  to   describe  each  risk  criterion  and  the  model  used  to  relate  the  potential  for  a  flight-­‐critical  failure  to  each  risk   criterion.   A   description   of   this   process   specific   to   the   definition   of   Part   1309   regulations   for   civil   UAS   is   provided   in   the   next   section.   However,   it   is   important   to   note   that   the  same   fundamental   process   can   be   used   to   tailor   regulations   relating   to   any   aspect   of   UAS   regulation,   including   maintenance,   personnel   training   and   licensing,   and   software-­‐assurance   levels   [e.g.,   the   tailoring   of   software-­‐assurance   levels   defined   in   DO-­‐178B   (RTCA  1992)].  

3.7.1 Example  –  Tailoring  of  Part  1309  Regulations   7

Part   1309   regulations   [e.g.,   FAR   Part   23.1309   or   FAR   Part   25.1309   (FAR   2009)]   define   “average   failure   probability  objectives”  to  guide  the  design  of  a  system  or  the  modification  or  installation  of  parts  to  existing   systems.   These   objectives   are   expressed   as   allowable   qualitative/quantitative   probabilities   assigned   to   individual  failure  conditions  (FAA  1988,  2009).   With   the   exception   of   a   draft   kinetic   energy-­‐based   approach   proposed   by   the   Joint   Authorities   for   Rulemaking  on  UAS  (JARUS  2009),  the  default  approach  for  the  definition  of  Part  1309  regulations  for  UAS  is  to   assign   the   same   system-­‐failure-­‐probability   objectives   as   used   for   CPA   [c.f.,   NATO   (2009)].   This   approach   is   based  on  the  premise  that  an  equivalency  in  the  average  probability  of  failure  will  lead  to  an  equivalency  in   safety.   Reliability   does   not   directly   equate   to   safety   and   thus   these   approaches   fail   to   recognise   the   fundamental  distinction  that  the  primary  EOV  at  risk  are  no  longer  onboard  the  aircraft,  but  rather  are  external   to  the  UA.  In  contrast,  this  paper  advocates  the  tailoring  of  Part  1309  regulations  through  the  use  of  a  simple   risk   model   that   relates   safety   criteria   (measures   of   potential   loss   determined   for   CPA)   to   measures   of   the   potential  for  a  flight-­‐critical-­‐failure  event.  In  Equation  2,  this  relationship  is  expressed  as  the  potential  of  a  loss   event   (ELOSCPA),   which   is   measured   with   respect   to   each   of   the   ELOS   criteria   specified   in   Equation   1,   the   potential   occurrence   of   a   flight-­‐critical   failure   (FAILURE),   and   the   potential   for   loss   given   the   occurrence   of   a   flight-­‐critical-­‐failure  (LOSS  |  FAILURE).       ELOSCPA    =  FAILURE  x  LOSS|FAILURE     Equation  2     Examples   of   a   model-­‐based   approach   are   presented   by   McGeer   et   al.   (1999),   Grimsley   (2004),   Weibel   and   Hansman   (2004),   Clothier   and   Walker   (2006),   Clothier   et   al.   (2007),   and   Dalamagkidis   et   al.   (2008a,   2009).   These   existing   models   relate   the   expected   number   of   casualties   per   flight   hour   (a   measure   of   CR)   to   the   expected  number  of  failures  per  flight  hour.  It  is  recommended  that  similar  models  relating  measures  of  the  IR,   CR   and   SR   to   measures   of   system   reliability   also   be   developed.   Irrespective   of   the   particular   measure,   the   general  process  for  tailoring  average  failure  probability  objectives,  as  described  in  the  following  sections,  is  the   same.  

3.7.1.1 Specifying  the  ELOS  Objective  (ELOSCPA)   In   accordance   with   the   ELOS   objective,   the   Part   1309-­‐equivalent   regulations   for   UAS   must   ensure   a   minimum  level  of  safety  equivalent  to  that  exhibited  by  CPA.  This  equivalency  may  be  described  in  terms  of   the   specification   of   a   loss   outcome   and   a   measure   of   the   potential   of   realising   that   outcome   for   CPA   operations  (i.e.,  a  measure  of  risk).  For  example,  McGeer  et  al.  (1999),  Grimsley  (2004),  Weibel  and  Hansman   (2004),   Clothier   et   al.   (2006,   2007),   and   Dalamagkidis   et   al.   (2008a,   2009),   specify   equivalency   in   terms   of   a   single  measure  of  the  expected  number  of  casualties  per  flight  hour,  CECPA.   One  approach  to  specifying  this  benchmark  is  to  conduct  an  analysis  of  CPA  accident  databases,  e.g.,  as   presented   by   Weibel   and   Hansman   (2004)   and   Clothier   and   Walker   (2006).   These   figures   represent   criteria   aggregated   across   the   entire   fleet   of   CPA.   As   described   in   paragraph   13.c   of   AC-­‐23.1309-­‐1D   (FAA   2009),   processes  are  needed  to  disaggregate  these  criteria  to  individual  aircraft  and  then  to  a  finite  number  of  failure   conditions.   The   outcome   of   these   processes   is   a   set   of   measures   of   IR,   CR   and   SR,   normalised   to   individual   aircraft-­‐failure  conditions.  

                                                                                                                                    Although   most   commonly   referred   to   as   “Part   1309”,   these   regulations   are   actually   sections   of   the   regulations  contained  in   Parts  23,  25,  27  and  29  (e.g.,  FAR  Part  23.1309,  FAR  Part  25.1309).  To  save  confusion   with  other  usage  of  the  term  “section”,  in  this  paper  these  regulations  are  referred  to  as  “Part  1309”. 7

©  Copyright  2011  Reece  Clothier  

 

Page  19  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

3.7.1.2 Modelling  the  Occurrence  of  a  Flight-­‐Critical  Failure  (FAILURE)   A   model   for   describing   the   occurrence   of   flight-­‐critical   failures   is   needed.   The   most   common   approach   (Grimsley   2004;   Weibel   and   Hansman   2004;   Clothier   et   al.   2007;   Clothier   et   al.   2008;   Dalamagkidis   et   al.   2008a,   2009)   is   to   assume   that   the   occurrence   of   failure   events   may   be   modelled   by   an   exponential   distribution   with   a   constant   rate   parameter,   λ   (usually   expressed   as   the   expected   number   of   failures   per   flight   hour  or  per  mission).  A  large  number  of  complex  factors  challenge  the  assumption  of  a  constant  failure  rate.   For   example,   the   rate   of   failure   will   change   with   increased   experience   in   the   operation   of   a   given   UAS   type   [e.g.,  the  burn-­‐in  or  infant-­‐mortality  period  evident  in  Figure  A-­‐2  of  OSD  (2009)]  and  will  depend  on  the  type  of   system   component   being   modelled   (e.g.,   crew   are   more   likely   to   fail   if   fatigued   or   stressed)   and   on   aspects   specific  to  an  operation  (e.g.,  manoeuvring  flight  or  weather  conditions).  However,  within  the  context  of  Part   1309   regulations,   the   output   needs   to   be   the   average   failure   rate   (referred   to   as   an   average   failure   probability),  which  is  equal  to  λ,  for  small  values  of  λ.  

3.7.1.3 Potential  for  Loss  Given  a  Flight-­‐Critical  Failure  (LOSS  |  FAILURE)   The  second  component  of  a  model-­‐based  approach  is  the  specification  of  the  potential  for  loss  given  the   occurrence  of  a  flight-­‐critical  failure.  This  is  a  complex  model  that  may  be  broken  into  sub-­‐models  describing:   the  potential  location  of  impact,  the  conditions  on  impact  for  a  given  type  of  flight-­‐critical  failure,  the  exposure   and  distribution  of  EOV,  the  stress  incident  on  EOV,  and  the  strength-­‐response  of  EOV  to  an  incident  stress.   McGeer   et   al.   (1999),   Grimsley   (2004),   Weibel   and   Hansman   (2004),   Clothier   et   al.   (2007,   2008),   and   Dalamagkidis   et   al.   (2008a,   2009)   present   simplified   models   for   determining   measures   of   the   casualty   expectation  associated  with  the  operation  of  UAS  over  populated  areas.  Similar  models  need  to  be  determined   for   each   of   the   ELOS   criteria   that   relate   the   occurrence   of   a   flight-­‐critical   failure   to   measures   of   the   IR,   CR,   and   profiles  describing  SR.  These  models  must  be  developed  for  each  type  of  loss  outcome  of  concern  (e.g.,  fatal   injury  of  people,  damage  to  property,  and  damage  to  the  environment).     The   evaluation   of   these   models   should   be   specific   to   each   operational   scenario.   For   example,   the   parameters  input  to  the  casualty-­‐expectation  model  presented  in  Clothier  et  al.  (2007)  include  the  population   density  and  the  dimensions  of  the  UA.  The  values  of  these  input  parameters  may  depend  on  the  particular  row   (i.e.,  the  operating  environment  over-­‐flown)  and  column  (i.e.,  type  category  of  UAS).  

3.7.1.4 Solving  for  the  Average  Failure-­‐Probability  Objectives   The   average   failure-­‐probability   objectives   for   UAS   may   be   determined   by   combining   Equation   1   and   2.   Specifically,   the   average   failure-­‐probability   objectives   may   be   determined   by   rearranging   and   solving   Equation   2  for  the  average  flight-­‐critical-­‐failure  rate  against  each  of  the  measures  used  to  describe  the  ELOS  objective   (i.e.,  IR,  CR,  and  the  profile  characterising  SR,  Equation  1).     A   conservative   management   approach   would   then   select   the   most   stringent   of   the   average   failure-­‐ probability  objectives  determined  for  each  certification  category.  This  defines  the  upper  limit  of  the  average   probability   per   flight   hour   for   failure   conditions   that   result   in   the   worst   loss   outcome   (e.g.,   defined   as   “catastrophic”   conditions   in   Part   1309   regulations).   Based   on   this   value,   average   probability   objectives   may   then  be  further  apportioned  to  failure  conditions  that  would  result  in  loss  outcomes  of  lower  concern.  Existing   definitions  of  failure  conditions  may  require  revision.  It  is  recommended  the  categories  of  failure  conditions  be   defined  based  on  the  degree  of  controllability  of  the  UA  given  the  failure  and  hence  the  ability  of  the  system  to   avoid  an  impact  in  an  inhabited  area.    

3.7.1.5 Summary   The  outcome  of  the  generalised  process  described  in  the  previous  sections  is  a  tailoring  of  the  Part  1309-­‐ equivalent  regulations  to  each  of  the  certification  categories  defined  in  the  certification  matrix.  The  approach   is   flexible   in   that   regulations   may   be   defined   in   consideration   of   the   diversity   of   systems   and   operational   environments.  The  approach  is  systematic  in  that  the  structure  of  the  risk  matrix  and  the  models  used  ensure  a   consistent  specification  of  regulations  in  accordance  with  the  degree  of  risk  associated  with  each  operational   scenario.  Finally,  the  approach  is  defensible  in  that  the   resultant  regulations  can  be  objectively  verified  against   the  overarching  requirement  for  an  ELOS  to  CPA.    

©  Copyright  2011  Reece  Clothier  

 

Page  20  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

3.8 Accommodating  Mitigation  Approaches   As   the   primary   risks   from   UAS   operations   are   to   EOV   external   to   the   system,   it   is   possible   for   UAS   to   employ  a  range  of  mitigation  strategies,  both  operational  and  technical,  to  reduce  the  levels  of  risk.  Examples   include   parachute   recovery   systems,   frangible   systems,   autonomous   recovery   and   autonomous   flight-­‐ termination   guidance   systems   [e.g.,   Mejias   et   al.   (2009)].   Within   the   airworthiness   framework   for   CPA,   such   mitigation  systems  are  addressed  on  a  case-­‐by-­‐case  basis  (e.g.,  the  parachute  flight-­‐termination  system  used   onboard   Cirrus   aircraft).   However,   for   UAS,   mitigation   systems   are   common;   hence,   a   systematic   method   of   incorporating  them  into  the  airworthiness  certification  framework  is  desirable.   Weibel   and   Hansman   (2005)   use   an   event-­‐tree   model   to   describe   the   effectiveness   of   mitigation   measures.  In  this  model,  mitigations  influence  the  probability  of:     1. entering  a  hazardous  state  (in  terms  of  formal  causal-­‐hazard  analysis  this  is  a  flight-­‐critical-­‐failure   state),   2. recovering  from  the  chain  of  failure  states  and  hence  prevention  of  the  immediate  realisation  of   the  hazard  state,     3. reducing  the  effects  of  the  mishap,  or     4. combinations  of  the  above.     Using   this   framework,   the   effectiveness   of   different   mitigation   strategies   may   be   systematically   defined   in   terms  of  permissible  movements  within  the  certification  matrix.  A  mitigation  strategy  may  be  assessed  as  to   whether  it  contributes  to:     1. a  reduction  in  the  plausible  level  of  loss  (i.e.,  representative  of  a  horizontal  movement  within  the   matrix  –  e.g.,  frangible  systems),     2. a   reduction   in   the   susceptibility   and   hence   potential   of   a   given   operational   area   to   register   loss   (i.e.,  representative  of  a  vertical  movement  within  the  matrix  –  e.g.,  operating  only  at  night  when   most  people  are  indoors  and  sheltered),  or     3. a   combination   of   both   (i.e.,   representative   of   a   diagonal   movement   within   the   matrix   –   e.g.,   a   controlled   parachute   flight-­‐termination   system   that   reduces   kinetic   energy   and   may   be   used   to   control  where  the  UAS  comes  to  earth).       Mitigation   systems   may   permit   certification   in   a   lower   airworthiness   category,   greater   operational   freedom   for   a   given   certification   category,   or   a   combination   of   both.   It   is   important   to   note   that   technical   mitigation   systems   only   address   a   subset   of   the   possible   failure   conditions   leading   to   an   occurrence   of   the   primary  hazard  and  may  introduce  new  failure  conditions  into  the  system.  Hence,  standards  and  regulations   are   required   for   the   certification   of   technical   mitigation   systems.   A   mechanism   for   the   certification   of   mitigation   systems   could   be   provided   by   the   issuing   of   a   supplementary   type   certificate   (STC)   to   a   UAS.   The   STC  could  also  include  details  of  any  permissible  operational  dispensations.  

3.9 Summary   Unlike  in  the  CPA  certification  framework,  in  the  proposed  ACM,  the  type  category  alone  does  not  define   the  airworthiness  category.  Instead,  the  framework  may  be  defined  so  as  to  prescribe  airworthiness  standards   and  regulations  through  consideration  of  both  the  system  (type)  and  its  intended  operation  (environment).  A   single   UAS   type   may   therefore   be   certificated   in   one   or   more   airworthiness   categories.   Similarly,   a   single   airworthiness   category   may   be   applicable   to   more   than   one   type   category   of   UAS.   The   objective   tailoring   of   standards  and  regulations  to  the  airworthiness  categories  is  based  on  the  level  of  risk  and  the  need  to  satisfy   the  ELOS  objective.   It   is   likely   that   existing   CPA   prescriptive   codes   of   airworthiness   standards   and   regulations   (e.g.,   Part   23,   Part   25,   and   Part   27)   will   be   mandated   for   higher   categories   of   airworthiness   (i.e.,   airworthiness   categories   assigned   to   cells   in   the   lower-­‐right   corner   of   the   ACM   illustrated   in   Figure   3).   However,   airworthiness   categories  that  present  lower  relative  risk  (i.e.,  airworthiness  categories  assigned  to  cells  closer  to  the  upper-­‐ left   corner   of   the   matrix   illustrated   in   Figure   3)   would   necessitate   less   stringent   regulation   (e.g.,   regulation   under  a  safety-­‐target  approach).     Finally,   the   ACM   has   some   added   practical   advantages.   Firstly,   the   structure   is   easily   visualised,   thus   providing   a   simple   tool   for   conveying   airworthiness   requirements.   In   addition,   the   compartmentalised   ©  Copyright  2011  Reece  Clothier  

 

Page  21  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

structure   facilitates   the   phased   development   and   introduction   of   airworthiness   regulations.   Regulatory   development  efforts  could  therefore  be  prioritised  according  to  industry  needs  and  those  areas  of  immediate   risk  or  to  capitalise  on  the  availability  of  existing  standards  and  regulations.  

3.10 Challenges  to  the  ACM  Approach   Cox   (2008)   identifies   a   number   of   limitations   in   the   use   of   risk   matrices.   Although   the   mathematical   analysis  provided  by  Cox  is  based  on  the  more  restrictive  position  that  risk  is  measured  through  an  arithmetic   multiplication  of  its  components,  general  issues  may  be  identified  from  the  discussion  provided.     As  with  any  assessment  of  risk,  there  is  the  difficulty  of  incorporating  events  with  uncertain  outcomes.  As   described   in   Section   §3.2,   the   concept   of   uncertainty   has   dimensions   that   are   not   captured   in   a   typical   risk   matrix.  These  include  the  aleatory  uncertainties  associated  with  the  measure,  as  well  as  the  complex  epistemic   uncertainties  associated  with  the  model,  process  and  data  used  in  an  assessment,  regardless  of  whether  it  is   qualitative   or   quantitative.   An   additional   dimension   of   the   matrix   providing   a   mapping   of   the   broader   concept   of  uncertainty  in  the  assessment  could  be  envisaged  [insight  into  how  this  could  be  structured  is  provided  by   Stirling  et  al.  (1998),  Figure  3].     Cox  (2008)  also  states  that  personal  judgements  and  the  potential  for  inconsistencies  “…implies  that  there   may   be   no   objectively   correct   way   to   fill   out   a   risk   matrix.”   All   assessments   of   risk   are   inherently   and   inescapably   subjective.   Such   subjectivities   are   not   specific   to   the   assessment   structure   provided   by   a   risk   matrix.   On   the   contrary,   risk   matrices   may   assist   in   distinguishing   between   the   sources   of   subjectivity   within   a   risk   assessment   and   hence   reduce   potential   stakeholder   conflicts.   For   example,   if   stakeholders   could   reach   agreement  on  the  specification  of  the  dimensions  of  the  matrix  then  the  subjectivities  lie  in  the  assessments  of   the   subsequent   quantification   of   the   loss   and   its   associated   measure   of   likelihood   or   probability.   An   encompassing  component  of  any  assessment  is  the  perspective  of  the  stakeholder.  Thus,  a  likely  input  to  any   decision-­‐making   process   is   a   set   of   matrices,   with   each   matrix   representing   a   particular   stakeholder’s   assessment   of   the   risks.   The   mapping   of   all   matrices   to   a   single   ACM   is   the   output   of   a   deliberative   and   subjective  process  involving  all  stakeholders.   Throughout   the   previous   section,   the   specification   of   a   matrix   structure   necessitated   that   a   number   of   trade-­‐offs  be  made.  These  primarily  relate  to  the  scope  of  assessment  and  the  resolution  of  the  dimensions.  In   addition  to  these  trade-­‐offs,  there  is  the  final  practical  issue  of  international  harmonisation  in  standards  and   regulations.   The   proposed   framework   facilitates   a   tailoring   of   regulations   to   the   industry,   operational   environment,   regulatory   needs,   and   the   political   and   social   demands   of   a   specific   nation.   For   example,   Australia  has  a  unique  operating  environment,  unique  applications,  and  a  unique  social  and  cultural  attitude   towards  aviation  technologies  (and  hence  risk  acceptance).  Although   the  matrix  approach  offers  flexibility  in   the   tailoring   of   airworthiness   regulations   to   a   specific   nation,   incompatibilities   may   arise   between   airworthiness   frameworks   developed   for   different   nations   and   hence   hinder   UAS   operations   in   international   airspace.   To   address   this,   regulators   could   either   seek   international   consensus   on   the   specification   of   the   matrix   or   could   define   compliance   matrices   that   provide   a   mapping   between   the   different   airworthiness   categorisation  schemes.      

4 Application  to  Airspace  Integration   The   risk   matrix-­‐approach   could   also   be   used   to   structure   regulations   governing   the   integration   of   UAS   operations  within  civil  airspace.     The   primary   hazard   governing   airspace   integration   is   that   of   a   midair   collision   with   another   aircraft   carrying   people.   A   sense-­‐and-­‐act   capability   equivalent   to,   or   better   than,   the   see-­‐and-­‐avoid   functionality   provided   by   a   human   pilot   is   viewed   as   one   the   most   significant   challenges   facing   the   non-­‐segregated   operation   of   UAS   within   the   NAS.   Despite   the   known   performance   limitations   of   a   pilot’s   see-­‐and-­‐avoid   capability   [see   ATSB   (1991)],   the   default   position   mandates   the   need   for   an   equivalent   functionality.   Equivalent   functionality   may   not   equate   to   equivalence   in   safety.   Requirements   on   the   operation   and   equipage  of  UAS  within  the  airspace  system  should  be  defined  in  consideration  of  the  entire  safety  case  [all   “layers”  (ARC  2009)]  and  not  solely  based  on  the  last  layer  of  defence  provided  by  a  human  pilot.   A   range   of   technologies   could   be   used   to   establish   a   safety   case   for   UAS   operations   in   the   civil   airspace   system.  It  is  proposed  that  the  fundamental  risk-­‐matrix  approach  could  provide  a  systematic  structure  for  the   assessment   of   such   safety   cases   and   the   development   of   operational   regulations   commensurate   with   the   levels   of   risk   presented   by   the   different   operations.   A   distinct   risk   matrix   could   be   structured   in   a   similar   fashion  to  that  illustrated  in  Figure  3  or,  if  the  UAS-­‐type  categories  used  were  the  same  as  those  defined  for   ©  Copyright  2011  Reece  Clothier  

 

Page  22  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

airworthiness   regulations,   then   the   application   of   the   risk-­‐matrix   approach   could   be   visualised   as   an   extension   of   the   existing   ACM   into   a   third   dimension   (Figure   5).   The   extra   dimension   would   correspond   to   the   categorisation  of  airspace  environments  based  on  the  potential  for  a  mid-­‐air  collision.  It  is  important  to  note   that   the   airspace   categories   illustrated   in   Figure   5   may   not   be   the   same   as   the   current   classes   of   airspace   defined  by  ICAO  (e.g.,  Classes  A-­‐G,  ICAO  2001).  The  level  of  service  provided  by  air-­‐traffic  services  is  but  one  of   many  factors  that  will  need  to  be  considered  in  the  definition  of  this  dimension  of  the  matrix.  These  factors   may  include:  radar  coverage;  the  distribution  and  number  of  aircraft;  the  level  of  pilot-­‐proficiency;  equipage   and  type  of  other  airspace  users;  density  of  airspace  users;  and  typical  meteorological  conditions.  For  example,   “Cat   A”   illustrated   in   Figure   5   could   represent   segregated   airspace   (e.g.,   a   prohibited   area);   “Cat   B”   could   represent  a  “managed”  and  “known”  airspace  environment  with  a  low  number  and  density  of  airspace  users.   At  the  other  end  of  the  spectrum  of  airspace  environments,  “Cat  Z”  could  represent  airspace  that  is  completely   unmanaged,   has   a   high   number   and   complex   mix   of   airspace   users,   and   includes   airspace   users   with   no   additional  situational  awareness  other  than  that  provided  by  an  onboard  pilot.  This  would  represent  airspace   of  greatest  potential  for  mid-­‐air  collision.     The   existing   UAS-­‐type   categories   could   also   be   grouped   into   those   considered   capable   of   causing   flight-­‐ critical   damage   to   another   aircraft   and   those   that   are   unlikely   to   cause   substantial   damage   to   other   aircraft   (i.e.,  micro  or  highly  frangible  UAS).  This  partitioning,  if  desired,  could  be  based  on  the  energy  limits  used  for   certifying  the  resilience  of  an  aircraft  empennage  or  propeller  to  a  bird-­‐strike  (defined  in  FAR  25.631  and  FAR   35.36,  respectively).   Within   Figure   5   each   three-­‐dimensional   cell   would   prescribe   the   airworthiness   and   operational   requirements   proportionate   to   the   degree   of   risk   that   a   given   UAS   type   presents   to   EOV   over-­‐flown   and   to   other   airspace   users,   respectively.   A   COA   could   then   be   issued   inclusive   of   the   systems   necessary   for   operations  in  the  given  airspace  environment.  For  example,  radios,  transponders,  sense-­‐and-­‐act  systems,  and   navigation  systems  can  be  considered  part  of  the  airworthiness  certification  baseline.   Airspace   integration   is   a   highly   politicised   issue,   and   hence   there   are   likely   to   be   many   external   factors   influencing   the   setting   of   regulations   on   UAS   integration   into   the   civil   airspace   system.   However,   the   application  of  the  proposed  risk-­‐matrix  structure  may  aid  the  resolution  of  such  discussions  by  providing  risk-­‐ informed   and   justifiable   boundaries,   within   which   further   rational   and   risk-­‐informed   discussion   may   take   place.  The  further  application  of  the  risk-­‐matrix  approach  to  the  issue  of  UAS  airspace  integration  is  the  subject   of  a  future  paper.    

  Figure  5  –  Illustration  of  a  combined  airspace-­‐integration  and  airworthiness  certification  strategy  for   UAS    

©  Copyright  2011  Reece  Clothier  

 

Page  23  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

5 Conclusion   UAS  are  a  fundamentally  new  aviation  technology  that  promises  many  benefits  to  society;  however,  they   also   represent   a   fundamentally   new   risk   paradigm   that   must   be   appropriately   managed.   A   number   of   limitations  associated  with  the  direct  adoption  of  the  existing  civil  CPA  airworthiness  framework  to  UAS  have   been  described.  These  limitations  justify  the  need  to  explore  possible  alternatives.  The  objective  of  this  paper   was   to   describe   an   approach   that   could   be   adopted   to   assist   in   the   structuring   of   the   airworthiness   requirements  for  UAS.  For  civil  UAS,  this  approach  could  be  used  to  structure  a  Part  21-­‐equivalent  regulation.   The   proposed   airworthiness   certification   matrix   (ACM)   offers   flexibility   by   allowing   regulations   to   be   tailored   in   consideration   of   the   levels   of   risk,   the   practical   and   commercial   limits   of   the   technology,   and   the   social   and   political   environment   in   which   the   regulatory   decisions   are   made.   In   addition,   through   the   quantified  specification  of  the  framework  it  is  possible  to  establish  a  transparent  and  justifiable  basis  in  terms   of  the  overarching  requirement  for  an  ELOS.  It  is  acknowledged  that  the  approach  is  not  without  its  challenges,   however,   as   stated   by   Bruce   Tarbert,   recently   retired   head   of   national   airspace   integration   FAA,   AIR-­‐160   Program  Office,  in  an  interview  regarding  the  certification  of  small  UAS:        …  SUAS  (Small  UAS)  cannot  be  effectively  certified  for  airworthiness  like  other  aircraft  types  can…  We  need   to  think  differently,  well  outside  the  box…   (La  Franchi  2009)     The   basis   of   the   proposed   ACM   approach   does   not   venture   far   beyond   the   boundaries   of   the   existing   “box”,  with  the  fundamental  principles  being  clearly  visible  in  existing  regulations  for  civil  CPA  (albeit  in  a  less   explicit   and   structured   manner).   The   systematic   structuring   of   the   problem   space   as   proposed   in   this   paper,   at   a  minimum,  will  aid  further  discussions  on  the  development  of  regulations  for  civil  or  commercially  operated   UAS.   The  practical  specification  of  the  ACM  for  the  regulation  of  civil  UAS  in  Australia  is  being  explored  by  the   Australian   Aerospace   Industry   Forum   (AAIF)   Certification   and   Regulation   Working   Group   Sub-­‐committee   on   UAS  Regulations.  In  May  2010  the  AAIF  Sub-­‐committee  provided  CASA  with  formal  recommendations  on  the   development  of  regulations  for  civil  UAS.  The  first  recommendation  was  that  the  ACM  approach  be  adopted  as   a  suitable  structure  for  the  airworthiness  certification  of  civil  UAS  in  Australia.    

Acknowledgements     The   authors   would   like   to   acknowledge   the   feedback   provided   by   Mr   Michael   Nas   (Murdoch   University,   Western   Australia),   Mr   Nicholas   Brewer   (CAA-­‐UK)   and   the   representatives   from   CASA,   the   UAS   industry   and   the   Australian   Department   of   Defence   participating   in   the   AAIF   Certification   and   Regulation   Working   Group   Sub-­‐committee  on  UAS  Regulations.  All  photographs  used  in  figures  are  in  the  public  domain.  This  research  is   supported,  in  part,  by  a  Queensland  State  Government  Smart  State  PhD  Scholarship,  the  Australian  Research   Council's  Linkage  Projects  funding  scheme  (project  number  LP100100302)  and  the  Smart  Skies  Project,  which  is   funded,  in  part,  by  the  Queensland  State  Government  Smart  State  Funding  Scheme.    

©  Copyright  2011  Reece  Clothier  

 

Page  24  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

Bibliography     ADF,  2002.  Defence  Instruction  (General)  OPS  02–2,  Australian  Defence  Force  Airworthiness  Management.   Australian  Department  of  Defence,  Canberra,  ACT,  Australia.   ARC,  2009.  Comprehensive  Set  of  Recommendations  for  sUAS  Regulatory  Development.  Small  Unmanned   Aircraft  System  Aviation  Rulemaking  Committee.   http://www.faa.gov/about/office_org/headquarters_offices/avs/offices/air/hq/engineering/uapo/r ulemaking/media/sUAS_ARC_Recs.pdf  (Mar.  27,  2010).   AS/NZS  ISO  31000:2009,  Australian/New  Zealand  Standard,  Risk  Management  –  Principles  and  Guidelines.   Standards  Australia,  Standards  New  Zealand.   ATSB,  1991.  Limitations  of  the  See-­‐and-­‐Avoid  Principle,  ATSB  Research  Report.  Australian  Transport  Safety   Bureau  (ATSB),  Canberra,  ACT,  Australia.   http://www.atsb.gov.au/media/32918/limit_see_avoid.pdf  (Mar.  27,  2010).   CAA,  2008.  Civil  Aviation  Publication  CAP-­‐722,  Unmanned  Aerial  Vehicle  Operations  in  UK  Airspace  –  Guidance.   London  UK  Civil  Aviation  Authority  (CAA),  Department  for  Transport  (DfT),  London,  UK.   CASA,  2000.  Advisory  Circular  AC  21.1(1),  Aircraft  Airworthiness  Certification  Categories  and  Designations   Explained.  Civil  Aviation  Safety  Authority  (CASA),  Canberra,  ACT,  Australia.   CASA,  2003.  Civil  Aviation  Safety  Regulations  1998  (CASR)  Part  21,  Certification  and  Airworthiness   Requirements  for  Aircraft  and  Parts.  Civil  Aviation  Safety  Authority  (CASA),  Canberra,  ACT,  Australia.   CASA,  2004.  Civil  Aviation  Safety  Regulations  1998  (CASR)  Part  101,  Unmanned  Aircraft  and  Rocket  Operations.   Civil  Aviation  Safety  Authority  (CASA),  Canberra,  ACT,  Australia.   CASA,  2006.  AC  21-­‐43(0),  Experimental  Certificate  for  Large  Unmanned  Aerial  Vehicle  (UAV).  Civil  Aviation   Safety  Authority  (CASA),  Canberra,  ACT,  Australia.   Clothier  R.,  Walker  R.,  2006.  Determination  and  Evaluation  of  UAV  Safety  Objectives.  Proceedings  of  the  21st   International  Unmanned  Air  Vehicle  Systems  (UAVS)  Conference,  Bristol,  United  Kingdom.     Clothier  R.,  Walker  R.,  Fulton  N.,  Campbell  D.,  2007.  A  Casualty  Risk  Analysis  for  Unmanned  Aerial  System   (UAS)  Operations  over  Inhabited  Areas.  Proceedings  of  the  Twelfth  Australian  International   Aerospace  Congress,  2nd  Australasian  Unmanned  Vehicles  Conference,  Melbourne,  Victoria,   Australia.     Clothier,  R.A.,  Fulton,  N.L.,  Walker,  R.A.,  2008.  Pilotless  Aircraft:  The  Horseless  Carriage  of  the  Twenty-­‐First   Century?  Journal  of  Risk  Research  11(8),  999-­‐1023.   Clothier  R.A.,  Palmer,  J.L.,  Fulton  N.L.,  Walker  R.A.,  2010.  Definition  of  airworthiness  categories  for  civil   Unmanned  Aircraft  Systems  (UAS).  Proceedings  of  the  7th  International  Congress  of  the   Aeronautical  Sciences  (ICAS),  Nice,  France.   Clothier,  R.A.,  Fulton,  N.L.,  Walker,  R.A.,  2011.  On  the  Definition  of  a  Safe  System.  Unpublished  manuscript.   Cox,  A.J.,  2008.  What's  Wrong  with  Risk  Matrices?  Risk  Analysis  28(2),  497-­‐512.   Dalamagkidis  K.,  Valavanis,  K.P.,  Piegl,  L.A.,  2008a.  On  Unmanned  Aircraft  Systems  Issues,  Challenges  and   Operational  Restrictions  Preventing  Integration  into  the  National  Airspace  System.  Progress  in   Aerospace  Sciences  44(7-­‐8),  503-­‐519.   ©  Copyright  2011  Reece  Clothier  

 

Page  25  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

Dalamagkidis  K.,  Valavanis,  K.P.,  Piegl,  L.A.,  2008b.  A  Survey  of  Unmanned  Aircraft  Systems  Regulation:  Status   and  Future  Perspectives.  Proceedings  of  the  16th  Mediterranean  Conference  on  Control  and   Automation,  Ajaccio,  France.  pp.717-­‐723.   Dalamagkidis  K.,  Valavanis,  K.P.,  Piegl,  L.A.,  2009.  On  Integrating  Unmanned  Aircraft  Systems  into  the  National   Airspace  System.  Springer, New  York  /  Heidelberg.

 

 

DeGarmo,  M.T.,  2004.  Issues  Concerning  Integration  of  Unmanned  Aerial  Vehicles  in  Civil  Airspace.  MITRE,   Center  for  Advanced  Aviation  System  Development,  McLean,  Virginia,  United  States  of  America.   DoD,  2000.  MIL-­‐STD-­‐882D,  Standard  Practice  For  System  Safety.  United  States  of  America  Department  of   Defense,  Washington,  DC,  United  States  of  America.   FAA,  1988.  Advisory  Circular  25.1309-­‐1A,  System  Design  and  Analysis.  Federal  Aviation  Administration  (FAA),   U.S.  Department  of  Transportation,  Washington,  DC,  United  States  of  America.   FAA,  2004a.  AC  20-­‐65A,  U.S.  Airworthiness  Certificates  and  Authorizations  for  Operation  Of  Domestic  and   Foreign  Aircraft.  Federal  Aviation  Administration  (FAA),  U.S.  Department  of  Transportation,   Washington,  DC,  United  States  of  America.   FAA,  2004b.  Order  8130.2F,  Airworthiness  Certification  of  Aircraft  and  Related  Products.  Federal  Aviation   Administration  (FAA),  U.S.  Department  of  Transportation,  Washington,  DC,  United  States  of   America.   FAA,  2009.  Advisory  Circular  23.1309-­‐1D,  System  Safety  Analysis  and  Assessment  for  Part  23  Airplanes.  Federal   Aviation  Administration  (FAA),  U.S.  Department  of  Transportation,  Washington,  DC,  United  States   of  America.   FAR,  2009.  14  CFR  FAR,  Title  14,  Code  of  Federal  Regulations,  Federal  Aviation  Regulations.  Federal  Aviation   Administration  (FAA),  U.S.  Department  of  Transportation,  Washington,  DC,  United  States  of   America.   Fischhoff  B.,  Slovic  P.,  Lichtenstein  S.,  Read  S.,  Combs  S.,  1978.  How  Safe  is  Safe  Enough?  A  Psychometric  Study   towards  Technological  Risks  and  Benefits.  Policy  Sciences  9(2),  127-­‐152.   Grimsley  F.,  2004.  Equivalent  Safety  Analysis  Using  Casualty  Expectation  Approach  (AIAA-­‐2004-­‐6428).   Proceedings  of  the  AIAA  3rd  Unmanned  Unlimited  Technical  Conference,  Workshop  and  Exhibit,   Chicago,  Illinois,  United  States  of  America.     Haddon,  D.R.,  Whittaker,  C.J.,  2002.  Aircraft  Airworthiness  Standards  for  Civil  UAVs.  UK  Civil  Aviation  Authority   (CAA),  London,  United  Kingdom.   Hayhurst,  K.J.,  Maddalon,  J.M.,  Miner,  P.S.,  DeWalt,  M.P.,  McCormick,  F.G.,  2006.  Unmanned  Aircraft  Hazards   and  Their  Implications  for  Regulation.  Proceedings  of  the  IEEE/AIAA  25th  Digital  Avionics  Systems   Conference  (DASC),  Portland,  OR,  United  States  of  America,  pp.1-­‐12.   ICAO,  2000.  Convention  on  International  Civil  Aviation,  Eighth  Edition.  International  Civil  Aviation  Organization   (ICAO).   ICAO,  2001.  Annex  11  to  the  Convention  on  International  Civil  Aviation,  Air  Traffic  Services,  Thirteenth  Edition.   International  Civil  Aviation  Organization  (ICAO).     ICAO,  2005.  Annex  8  to  the  Convention  on  International  Civil  Aviation,  Airworthiness  of  Aircraft,  Tenth  Edition.   International  Civil  Aviation  Organization  (ICAO).  

©  Copyright  2011  Reece  Clothier  

 

Page  26  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

ISO  Guide  73:2009.  International  Organization  for  Standardization  (ISO),  Risk  Management  –  Vocabulary.  ISO,   Switzerland.   JAA,  2004.  UAV  Task-­‐Force  Final  Report,  A  Concept  for  European  Regulations  For  Civil  Unmanned  Aerial   Vehicles  (UAVs).  The  Joint  JAA/EUROCONTROL  Initiative  on  UAVs.   http://www.easa.eu.int/ws_prod/r/doc/NPA/NPA_16_2005_Appendix.pdf  (Mar.  26,  2010).   JARUS,  2009.  JARUS  Working  Paper  on  UAS  System  Safety  for  Airworthiness  (Draft).  Joint  Authorities  for   Rulemaking  on  UAS  (JARUS).   Jongejan,  R.B.,  Ale,  B.J.M.,  Vrijling,  J.K.,  2006.  FN-­‐Criteria  for  Risk  Regulation  and  Probabilistic  Design.   Proceedings  of  the  International  Conference  on  Probabilistic  Safety  Assessment  and  Management   (PSAM),  New  Orleans,  Louisiana,  USA.     Kaplan  S.,  Haimes,  Y.Y.,  Garrick,  B.J.,  2001.  Fitting  Hierarchical  Holographic  Modeling  into  the  Theory  of   Scenario  Structuring  and  a  Resulting  Refinement  to  the  Quantitative  Definition  of  Risk.  Risk  Analysis   21(5),  807-­‐819.   Klinke  A.,  Renn  O.,  2002.  A  New  Approach  to  Risk  Evaluation  and  Management:  Risk-­‐Based,  Precaution-­‐Based,   and  Discourse-­‐Based  Strategies.  Risk  Analysis  22(6),  1071-­‐1094.   La  Franchi  P.,  2009.  The  Long  Road  to  Integration.  Unmanned  Vehicles  14(4),  46-­‐51.   Lin  M.,  Larson  E.,  Collins  J.,  2003.  Volume  II,  Appendix  D.16,  Determination  of  Debris  Risk  to  the  Public  Due  to   the  Columbia  Breakup  during  Reentry.  Columbia  Accident  Investigation  Board  (CAIB).   http://www.klabs.org/richcontent/Reports/Failure_Reports/columbia/caib_report/volume_2/part1 6.pdf  (Mar.  26,  2010).     Luxhøj,  J.T.,  2009.  Safety  Risk  Analysis  of  Unmanned  Aircraft  Systems  Integration  Into  the  National  Airspace   System:  Phase  1.  Federal  Aviation  Administration  (FAA),  Air  Traffic  Organization,  NextGen  &   Operations  Planning,  Office  of  Research  and  Technology  Development,  Washington,  DC,  United   States  of  America.   McGeer  T.,  2007.  Safety,  Economy,  Reliability,  and  Regulatory  Policy  for  Unmanned  Aircraft.  Aerovel   Corporation,  White  Salmon,  WA,  USA.   http://www.aerovelco.com/papers/RoboticAircraftSafetyAndEconomy.pdf  (Mar.  26,  10)   McGeer  T.,  Newcome  L.,  Vagners,  J.,  1999.  Quantitative  Risk  Management  as  a  Regulatory  Approach  to  Civil   UAVs.  Proceedings  of  the  Second  Annual  European  Unmanned  Vehicle  Systems  Association   Conference  on  UAV  Certification,  Paris,  June.     McGeer  T.,  Vagners  J.,  1999.  Wide-­‐Scale  Use  of  Long-­‐Range  Miniature  Aerosondes  over  the  World's  Oceans.     The  Insitu  Group.  Bingen,  WA,  USA.   Mejias  L.,  Fitzgerald,  D.L.,  Eng,  P.C.,  Xi  L.,  2009.  Forced  Landing  Technologies  for  Unmanned  Aerial  Vehicles:   Towards  Safer  Operations.  In:  Thanh  Mung,  L.,  (Eds.),  Aerial  Vehicles.  In-­‐Tech.  Kirchengasse,   Austria.     Nas  M.,  2011.  Classifying  Unmanned  Aircraft  Systems:  Development  of  an  Objective  Framework  for  Evaluating   UAS  Classification  Schemes.  Masters  Thesis,  Murdoch  University,  Perth,  WA,  Australia.   NATO,  2009.  STANAG  4671,  Standardization  Agreement,  Unmanned  Aerial  Vehicles  Systems  Airworthiness   Requirements  (USAR).  North  Atlantic  Treaty  Organization  (NATO)  Standardization  Agency,  Brussels,   Belgium.   OSD,  2007.  Unmanned  Systems  Roadmap,  2007–2032.  United  States  Department  of  Defense,  Office  of  the   Secretary  of  Defense,  Washington  D.C.,  USA.     ©  Copyright  2011  Reece  Clothier  

 

Page  27  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

OSD,  2009.  Unmanned  Systems  Integrated  Roadmap  (2009–2034).  United  States  Department  of  Defense,   Office  of  the  Secretary  of  Defense,  Washington  D.C.,  USA.   Paté-­‐Cornell  E.,  1994.  Quantitative  Safety  Goals  for  Risk  Management  of  Industrial  Facilities.  Structural  Safety   13(3),  145-­‐157.   RCC,  2007.  Standard  321-­‐07,  Common  Risk  Criteria  Standards  for  National  Test  Ranges.  Range  Commanders   Council,  United  States  Department  of  Defense,  New  Mexico,  USA.   RTCA,  1992.  DO-­‐178B,  Software  Considerations  in  Airborne  Systems  and  Equipment  Certification.  RTCA,   Washington  D.C.,  USA.   RTCA,  2007.  DO-­‐304,  Guidance  Material  and  Considerations  for  Unmanned  Aircraft  Systems.  RTCA,   Washington  D.C.,  USA.   Stirling  A.,  1998.  Risk  at  a  Turning  Point?  Journal  of  Risk  Research  1(2),  97-­‐109.   Weibel,  R.E.,  Hansman,  R.J.,  2004.  Safety  Considerations  for  Operation  of  Different  Classes  of  UAVs  in  the  NAS   (AIAA-­‐2004-­‐6421).  Proceedings  of  the  American  Institute  of  Aeronautics  and  Astronautics  (AIAA),   3rd  Unmanned  Unlimited  Technical  Conference,  Workshop  and  Exhibit,  Chicago,  Illinois,  USA.     Weibel,  R.E.,  Hansman,  R.J.,  2005.  An  Integrated  Approach  to  Evaluating  Risk  Mitigation  Measures  for  UAV   Operational  Concepts  in  the  NAS  (AIAA-­‐2005-­‐6957).  Proceedings  of  the  American  Institute  of   Aeronautics  and  Astronautics  (AIAA),  4th  [email protected]  Conference,  Arlington,  VA,  USA.      

©  Copyright  2011  Reece  Clothier  

 

Page  28  of  29  

Definition  of  an  Airworthiness  Certification  Framework  for  Civil  Unmanned  Aircraft  Systems  

 

  Clothier,  RA.  et  al.  (2011)  

Glossary     AAIF     ACM     ADF       CASA     COA     CONOPS     CPA       CR       DSTO     ELOS     EOV     FAA       ICAO     IR       NAS       SR       STC       SUAS     UA       UAS            

Australian  Aerospace  Industry  Forum   Airworthiness  certification  matrix   Australian  Defence  Force   Civil  Aviation  Safety  Authority  (Australia)   Certificate  of  Airworthiness   CONcept  of  OPerationS     Conventionally  piloted  aircraft   Collective  risk   Defence  Science  and  Technology  Organisation   Equivalent  level  of  safety   Entities  of  value   Federal  Aviation  Administration   International  Civil  Aviation  Organization   Individual  risk     National  Airspace  System   Societal  risk   Supplementary  type  certificate   Small  unmanned  aircraft  system/s   Unmanned  aircraft   Unmanned  aircraft  system/s  

©  Copyright  2011  Reece  Clothier  

 

Page  29  of  29