NTSB Lithium-Ion Battery Forum reports

Discussion in 'General Discussion | Travel' started by jbcarioca, Apr 11, 2013.  |  Print Topic

  1. jbcarioca
    Original Member

    jbcarioca Gold Member

    Messages:
    17,507
    Likes Received:
    57,455
    Status Points:
    20,020
    I am attending the NTSB Forum today and tomorrow. The full transcripts and video will soon be available on the NTSB website.

    Today there was a panel of technical experts in design and production of Li-Ion batteries. The presentations were very interesting. In particular it was notable that one producer, Quallion, has installed roughly one million li-ion batteries during the last 20 years with zero service failures. They use the technology for powering heart pacemakers.

    From the research and academic sides came the flat statement that li-ion is now a mature technology, regardless of specific chemistry, and that in-service evens have been exceedingly rare. They discussed specific failures:
    1. the Chevy Volt accidents producing electrolyte leakage;
    2. the Fisker Karma linked to poor production quality control (the producer went bankrupt as well as
    did Fisker);
    3. A Navy submarine charger failure while in dock for service;
    4. Incidents/accidents from 1999 at LAX, 2006 UPS, and 2010 UPS in DXB
    Each of these was determined to have improper packaging, charging, design flaws and/or production quality control
    3. From the NASA representative came the flat statement that they have had minimal issues with the Mars Rover, Orbiter systems and satellites. The NASA points were absolute intolerance for quality lapses in cell production, including 100% checking at every point and complete documentation. NASA itself tests small format batteries on a sampling basis but large format 100%. Satellites have working individual batteries in use for more than 20 years.
    4. The two manufacturers (SAFT and Quallion) both emphasized the same points: careful design with physical separation between cells and both thermal and electrical protections, 100% testing of every single component for precise adherence to design, production in dry, clean isolated rooms with zero human physical contact with components, testing of every single process step and rigorous 100% testing of all large format cells, batteries, connectors, chargers, and multi-redundant controls to ensure rate of charge was not too high, and limit of charge on both high and low side.
    5. The academics provided very interesting data which in the end, after a specific NTSB question produced the following: Li-Ion batteries, properly designed and controlled, have an expected service life of 25 years or more without problems. The repeated case mentioned by both academics and manufacturers was the Daimler municipal busses that have now more than 400 million hours in normal service with zero battery failures.
    6. The SAFT representative repeatedly mentioned that the US Navy is the standard for Li-Ion acquisition, in that they had very precise standards for performance but no requirement for a specific chemistry.
    7. Every panelist agreed that the specific chemistry was not an issue, in that the most volatile chemistry can be made perfectly safe while the most stable chemistry can be unsafe. The then, all of them, repeated the mantra of excellence in design, absolute quality control of all inputs, absolute pristine production, checking each step, testing each step and testing the entire system rigorously.
    8. Other than as part of the transport, not use, transport of li-ions from B787's there was no overt mention of that aircraft. There was a significant discussion of li-ion transportation issues which i will nto discuss here.

    During breaks the clear consensus view of the Boeing B787 issues was that they involved three issues, and only three:
    1. Bad design of Li-Ion batteries, electrical controls, charging systems, use systems and charging/discharge protocols. From one panelist seeking anonymity "we've known how to do this correctly for 15 years. No one understands how this could have happened if Boeing, Thales or GS Yuasa had followed known processes.
    2. Poor production quality control by GS Yuasa followed by inadequate testing by not only them but Thales and Boeing.
    3. Inadequate monitoring and control of battery charge, discharge and use processes.

    The final point of the day was from the RTCA representative. RTCA is the organ used by the FAA to develop new standards, which they do through a collaborative process with all parties from users, manufacturers and regulators to ensure that they can establish sound minimum Performance Stabdards for anything new that will be attached to an airplane. Boeing itself chaired the committee on use of Li-Ion batteries installed in aircraft in 2008. Later Boeing said the regulations had not been available when the B787 was designed, which was true but incomplete. Actually Boeing had made the request for a standard from the FAA, which ended out with them chairing the committee. Although the Boeing statement is true, all parties to the design at Boeing, including GS Yuasa, were well aware of the design principles and testing procedures used at the time.

    Why did Boeing ignore their own knowledge from the RTCA proceedings? We may never know, but the NTSB Boeing B787 hearings later this month should be fascinating!

    I will update tomorrow's meeting soon after the clsoe and will post the links to the further information as soon as they become available
     
  2. Seacarl
    Original Member

    Seacarl Gold Member

    Messages:
    10,521
    Likes Received:
    11,372
    Status Points:
    16,520
    Thanks for the excellent report of the hearings. For me it calls into question whether the Boeing solution is the right solution. Boeing's solution seems focused on containing a battery fire, with a minimal improvement to the battery, but without adequately addressing the three points you cite:
     
  3. secretsea18
    Original Member

    secretsea18 Gold Member

    Messages:
    7,729
    Likes Received:
    44,584
    Status Points:
    14,020
    Thank you for making this summary if the proceedings. .
     
    uggboy, IDGflygirl, boondr and 2 others like this.
  4. jbcarioca
    Original Member

    jbcarioca Gold Member

    Messages:
    17,507
    Likes Received:
    57,455
    Status Points:
    20,020
    Actually, Boeing seems to have addressed all the battery-specific issues, including physical separation of cells, venting of excess electrolyte, individual cell monitoring, individual cell connectors and disconnectors, charging, discharging and rate of charge limits. We have no data that I know regarding the integrity of the electrical system, post redesign of some parts, nor of the specific battery controller processes. We also do not know about the testing and production control standards being used now.

    On the whole I suspect Boeing has probably done a fairly complete job from a safety standpoint. Beyond that I do not have a view.

    OTOH, I'm thinking of returning to DC for the next hearing. It ought to be quite an event.
     
    uggboy, IDGflygirl, milchap and 4 others like this.
  5. euromannn
    Original Member

    euromannn Gold Member

    Messages:
    2,620
    Likes Received:
    8,619
    Status Points:
    7,070
    "careful design with physical separation between cells and both thermal and electrical protections, 100% testing of every single component for precise adherence to design,"

    Good design practice.

    Thales Battery Monitoring SYstem is used by Airbus as well as Boeing.
    Battery was the issue from day one.
     
    uggboy, boondr and jbcarioca like this.
  6. MSPeconomist
    Original Member

    MSPeconomist Gold Member

    Messages:
    58,563
    Likes Received:
    98,528
    Status Points:
    20,020
    It sounds to me like the manufacture of these batteries fails to exhibit some very minimal robustness criteria. Needing to test 100%, have no human handling, etc. would seem difficult tomrealistically satisfy in practice, which would make me question the design decision to make such batteries a crucial part of the aircraft design. It's one thing to make and handle the batteries to exacting standards for the Mars Rover, but very diffent to satisfy such precise standards in a commercial aircraft use and maintenance environment.
     
    uggboy, milchap, LETTERBOY and 2 others like this.
  7. jbcarioca
    Original Member

    jbcarioca Gold Member

    Messages:
    17,507
    Likes Received:
    57,455
    Status Points:
    20,020
    Those are the standards currently in place for production of volumes in excess of 100,000 large format units per year, according to the manufacturers today. It is not excessive, according to SAFT, who say it is simple economics that justify it because the cost of scrap is so high? Safety is an added benefit.

    Sent from my iPad using milepoint
     
    bonnerbl, uggboy, milchap and 2 others like this.
  8. jbcarioca
    Original Member

    jbcarioca Gold Member

    Messages:
    17,507
    Likes Received:
    57,455
    Status Points:
    20,020
    You may be correct, but airframer says Thales only supplies the power conversion component ray and controllers are supplied by UTC. It may be incorrect or I may be misreading it.


    http://www.airframer.com/aircraft_detail.html?model=A350

    In any event, Thales is also highly experienced. However, the design us a major issue and as all evidence clearly indicates the battery is. Major part of the problem, but absolutely not the entire problem. The discharge process, charging process, charge limits, discharge limits were also factors without which the failures probably would not have happened. I am not an expert, but I certainly am hearing the systems management story very distinctly today.

    Sent from my iPad using milepoint
     
    bez7, uggboy and boondr like this.
  9. NYCUA1K

    NYCUA1K Gold Member

    Messages:
    4,739
    Likes Received:
    5,086
    Status Points:
    4,270
    Great summary of the NTSB proceedings on the Li-ion battery, JBC. Much appreciated. As for what Boeing may have done, I do agree that without pinpointing or knowing the root cause, "battery-specific issues, including physical separation of cells, venting of excess electrolyte, individual cell monitoring, individual cell connectors and disconnectors, charging, discharging and rate of charge limits" must be reviewed and tested. So I am glad that Boeing appears to have done just that...
     
    LETTERBOY, uggboy and jbcarioca like this.
  10. milchap
    Original Member

    milchap Gold Member

    Messages:
    27,636
    Likes Received:
    148,038
    Status Points:
    20,020
    Thanks jbc for the report. :cool:
     
    LETTERBOY, jbcarioca and uggboy like this.
  11. Seacarl
    Original Member

    Seacarl Gold Member

    Messages:
    10,521
    Likes Received:
    11,372
    Status Points:
    16,520
    Your points were:

    1. Bad design of Li-Ion batteries, electrical controls, charging systems, use systems and charging/discharge protocols. From one panelist seeking anonymity "we've known how to do this correctly for 15 years. No one understands how this could have happened if Boeing, Thales or GS Yuasa had followed known processes.
    2. Poor production quality control by GS Yuasa followed by inadequate testing by not only them but Thales and Boeing.
    3. Inadequate monitoring and control of battery charge, discharge and use processes.

    Regarding the first point - I didn't think Boeing's redesign increased the separation of the cells as the battery size was fixed, but rather added a thin layer of an insulating material. And regarding the charging/discharging, while Boeing has tightened the limits, an issue is that all the cells are charged in series, not in parallel, and the cells, which are really individual batteries, are not individually monitored for charge levels. And that apparently, even when fully charged, there is some trickle charging that occurs because there are also some slight loads causing discharging. These points play into the 3rd point as well. I assume the reasons why greater physical separation and a redesign of the charging system haven't been addressed is that they require greater engineering and testing changes, and that what's been done is what can be done in the same physical rack space in the electrical bay and using the same electrical system... but that the battery experts have said those should have been designed differently

    I haven't heard much about improved quality control, your second point, but we can assume that will definitely occur.

    Please do and let us know!
     
  12. jbcarioca
    Original Member

    jbcarioca Gold Member

    Messages:
    17,507
    Likes Received:
    57,455
    Status Points:
    20,020
    secretsea18 and uggboy like this.
  13. jbcarioca
    Original Member

    jbcarioca Gold Member

    Messages:
    17,507
    Likes Received:
    57,455
    Status Points:
    20,020
    Charge level of cells connected in series will be, by definition, equal, as you say, unless; stratification within one or more cells prevents the individual cell from charging evenly or a physical anomoly causes some sort of short. Both of those things can cause unevenness in total charge within a series connected group of cells. The first problem, if I understand correctly, is most common in large format cells. the latter could be any size cells and normally results from some manufacturing defect or assembly error.

    I still do not know for certain that the B787 battery architecture has either charge or discharge occurring under nominally fully charged conditions. That was true with non-Li-ion Boeing electric, but the people I talk with are divided about the B787 situation. Soon I'm sure we'll know for sure. The experts at the forum are united in saying that a continuous trickle charge of a fully charged li-ion battery will cause thermal runaway. According to them, the question is only when. I find it unimaginable that Boeing, Thales and GS Yuasa would all permit such a basic design flaw to persist.

    the unimaginable is not impossible, I concede.
     
    uggboy and LETTERBOY like this.
  14. Seacarl
    Original Member

    Seacarl Gold Member

    Messages:
    10,521
    Likes Received:
    11,372
    Status Points:
    16,520
    So when charging in series, absolute quality control and ensuring that each cell is identical is critical to the performance of the battery. I found it interesting that some of the experts talked about batteries that lasted 15 years while Boeing was already replacing many B787 batteries in the first year. Seems like they were sorely lacking on the needed quality control.
    I am very interested in finding out about the trickle charge/discharge in the full charged condition. It's really unimaginable, but it's also strange that this rumor hasn't been definitively addressed one way or the other in the investigation.
     
  15. jbcarioca
    Original Member

    jbcarioca Gold Member

    Messages:
    17,507
    Likes Received:
    57,455
    Status Points:
    20,020
    Notable quotations from today's session:
    Dr. Celina Mikolajczak, Manager, Cell quality at Tesla: with 60 million cells in service, half a trillion cells hours service they have not a one thermal runaway. "However, inevitably we will have such failures due to the huge number of cells we have in service". They test 100% of all cells and their manufacturers test 100% also, including recording individual cells performance at end of manufacturing. Tesla maintains that history online and adds data every day on every cell in service through wireless connection to the vehicles.
    They have had vehicles flooded in Hurricane Sandy; upon inspection and testing 100% cells survived. They have had exotically odd crashes (showing a photo of a crashed Tesla Roadster with one car crashed into the side of the car and a large SUV resting on top of the Tesla; the entire battery pack was unscathed.

    The NHTSA records the Chevy Volt crash test in which the battery pack caught fire 23 days after the test. Chevy redesigned, no faults in service. The Fisker Karma had several fires of 123 batteries due to manufacturing and charging system flaws. Both Fisker and 123 are now defunct.

    All parties struggle to find other cases than the half dozen cited. The transportation of li-ions now has an excellent safety record after standards and controls were established and implemented.

    Over and over everyone from the US Navy to manufacturers to users favor small format batteries when possible due to easier risk control, better containment and reduced impact of failure.

    There are some applications demanding large format cells. There is unanimous agreement that there is no adequate basis to establish failure rates for large format cells, so the norm is to quote statistics for 18650 size cells, used in everything from Tesla to cellular telephones and produced in the billions. That is a size and form factor, not a specific chemistry or design. Tesla cells, for example, look like cellphone cells but have almost nothing in common with them. That size cell has about one failure in 10 million now, since they started in common use 20 years ago. Some current estimates are about 1 in 40 million, reflecting major improvements in design, materials and quality control during the two decades.

    A major surprise for me was to find that the US Navy has zero cells in service designed explicitly for them. They buy commercially available batteries, not just cells, but have very high performance standards. They do specify specific chemistries in some cases, but not all.

    There is copious data to absorb still. I have come from this Forum convinced that Li-Ion technology is mature, but quality is still improving as is performance. As one manufacturer, Saft, said: "we do not produce at state of the art. When we're tempted we always scale back enough to ensure that we know how the product will perform in service, and how to mitigate whatever failures may occur".

    From Tesla to the US Navy to Saft and Eagle Picher, all emphasize moving deliberately with incredible attention to detail at every step of the process, and testing, testing, testing from cell materials, to materials preparation, manufacturing, packaging, installation and management.

    As one participant said, "all that effort is worth it! Lithium-ion batteries are awesome! They require no maintenance, they can be cycled almost without limit, they last 40 years or more". Several manufacturers and end users say there are minimal failures in service with replacement rates very low. However, all the users and manufacturers measure their service conditions as if these things fail every day.
     
  16. jbcarioca
    Original Member

    jbcarioca Gold Member

    Messages:
    17,507
    Likes Received:
    57,455
    Status Points:
    20,020
    People went out of their way to not mention Boeing, Thales or GS Yuasa. Discussions in breaks made it crystal clear that there really is no excuse for the B787 situation. "Appalling lack of basic attention by everyone involved" is a quote I overheard from an influential participant who shall remain nameless.

    So are we all. Some specialist applications, such as some produced by Eagle Picher, have charge monitoring and charge controllers integrated with the individual battery, which does cause a trickle discharge. Those, however, are for very specific applications and do NOT involve trickle changing. Everyone repeated that trickle charing of a Li-ion causes thermal runaway.

    One comment. Treatment of ni-cads and Li-ions is very, very different. The normal aircraft battery of choice has been ni-cads for many years, now suddenly they're li-ion. That may be informative, maybe not.
     
  17. Seacarl
    Original Member

    Seacarl Gold Member

    Messages:
    10,521
    Likes Received:
    11,372
    Status Points:
    16,520
    Ultimately I believe that Lithium batteries can be made safe and will be made safe, but that Boeing needs to improve its implementation, which I believe they will do.

    However, I am a bit surprised at the cycled without limit, last 40 years quotation. My Li batteries in my laptop do not last much past two years before they are 50% degraded. When they cite numbers like 40 years, do they have very few charge/discharge cycles or very low rates, or is there something materially different about my laptop batteries?
     
  18. euromannn
    Original Member

    euromannn Gold Member

    Messages:
    2,620
    Likes Received:
    8,619
    Status Points:
    7,070
    "I still do not know for certain that the B787 battery architecture has either charge or discharge occurring under nominally fully carved conditions. That was true with non-Li-ion Boeing electric, but the people I talk with are divided about the B787 situation. Soon I'm sure we'll know for sure. The experts at the forum are united in saying that a continuous trickle charge of a fully charged li-ion battery will cause thermal runaway"

    1. What does fully carved conditions mean?
    2. Thermal runaway is a term that the NTSB and Boeing interpret differently so be careful when using it. Boeing has publicly stated that it has a different definition of thermal runaway versus how the NTSB uses it to describe the battery failure phenomena. NTSB then told Boeing it was inconsistent with it's team interpretation of thermal runaway. SInce this miscommunication neither has discussed thermal runaway in detail since.
     
    IDGflygirl, milchap and jbcarioca like this.
  19. jbcarioca
    Original Member

    jbcarioca Gold Member

    Messages:
    17,507
    Likes Received:
    57,455
    Status Points:
    20,020
    That was a typo for "charged"
    I fail to understand different interpretations of the term "thermal runaway". It is a well understood term in chemistry and physics and applies to tiny storage cells as it does to stellar masses. Boeing may choose a definition and the NTSB another if one chooses to apply it, for example to only a single cell, while another may prefer a string (parallel connected cells) or a pack (including series connected strings). The basic term is clear and simple. even the Wiki is decent on the subject, and all battery experts I have heard agree that if heat is added more quickly than it dissipates the rate of heating accelerates at an increasing rate which will result in uncontrolled heating until the available electrolyte has been consumed, thus, technically being a self limiting process. I listened to the entire board of the NTSB speaking on this subject, one by one and listened also to a number of industry participants and academics. The list is at the link given in post #12 in this thread. Precisely zero conflict existed about the definition of thermal runaway. That included the member from the RTCA. Boeing chaired the RTCA committee that produced DO-311, the standard now applied by the FAA. NTSB explicitly stated support for that standard. If anybody now can claim disagreement over such a basic definition I'd be surprised.

    IIRC, one area os disagreement was whether it is appropriate for Boeing to officially claim cell failure due to thermal runaway at rates of 1 in 10 million or more, while the history of such a number derives only from a non-analogous mass production small cell environment rather than the large cell environment Boeing deployed. No participant in the sessions claimed there was any plausible way to make any given estimate for large format cells because there is inadequate history with the format on which to base such prognostications. Such an issue could cause discord, particularly when the predicting party has observed failure rates vastly in excess of those levels.

    I am not taking a position about any such dissent because i am not aware of it, nor the details beyond one press report.
     
    euromannn, milchap and LETTERBOY like this.
  20. euromannn
    Original Member

    euromannn Gold Member

    Messages:
    2,620
    Likes Received:
    8,619
    Status Points:
    7,070

    I am not trying to debate thermal runaway just to note that Boeing has publicly stated it disagrees with NTSB interpretation.

    http://blogs.wsj.com/japanrealtime/2013/03/15/thermal-runaway-on-787-well-it-depends/

    "
    “I’d like to make one clarification if I could: thermal runaway is defined in many different ways,” Mike Sinnett, head of Boeing’s 787 program, said at a press conference Friday. “The definition of a thermal runaway depends on the perspective of the observer.”"
     
    jbcarioca likes this.
  21. mattsteg
    Original Member

    mattsteg Gold Member

    Messages:
    3,276
    Likes Received:
    5,543
    Status Points:
    4,170
    In industrial use, there is often more willingness to not fully charge or discharge the battery, and perhaps better control the climate as well as charge/discharge. With consumer electronics there is more pressure to use the extremes of the battery's range and be able to quote a longer lifetime per charge.

    The 40 year comment surprised me well, but batteries in consumer devices are speced to be abused, consumed, and discarded.
     
    jbcarioca likes this.
  22. euromannn
    Original Member

    euromannn Gold Member

    Messages:
    2,620
    Likes Received:
    8,619
    Status Points:
    7,070
    40 year battery life cycle must be a laboratory extrapolated guess as it's doubtful production usage would back up this individual's claim.
     
    mattsteg and jbcarioca like this.
  23. jbcarioca
    Original Member

    jbcarioca Gold Member

    Messages:
    17,507
    Likes Received:
    57,455
    Status Points:
    20,020
    Although the first commercial li-ion batteries were only produced in 1991 by Sony, the Li-Ion battery was 'invented' most correctly 'first proposed by Dr. M S Whittingham, a penelist at the NTSB discussion. He, his academic colleagues and the representatives of Saft and others all stated the 40 year expected life. Their statements, they said, were based on extrapolations from mass production 18650 cells produced in the early 1990's. Were the claims solid? I do not know. They all emphasized careful battery management and careful avoidance of maximum change and maximum discharge areas. They also emphasized that abuse of those cycles very likely result in an expanded IOS layer (if anybody cares that is explained in several of the presentations made in the NTSB Forum). Several reported that properly managed cells have been operating without incident in satellites for a couple fo decades.

    I find it very difficult to challenge the logic of these people. the ones who invented the technology, developed it for deployment in space, under water and in aircraft, trains, busses and cars were all these. When they all agree on a subject I am inclined to accept their answer as correct. They do not agree on all topics.
     
  24. jbcarioca
    Original Member

    jbcarioca Gold Member

    Messages:
    17,507
    Likes Received:
    57,455
    Status Points:
    20,020
    I do recall that press report. Mr Sinnett does not always get his facts correct, as we know, such as his assertion that the B787 batteries were not needed in flight. He definitely is not a battery expert. I suspect Boeing experts on the subject, if they exist, are anxious to keep their heads well down. You certainly have shown the quotation of Mr Sinnetts views.
     
    LETTERBOY likes this.
  25. mattsteg
    Original Member

    mattsteg Gold Member

    Messages:
    3,276
    Likes Received:
    5,543
    Status Points:
    4,170
    jbcarioca has far more detailed insight than I, but I agree that the 40 year claim is quite credible, in the right situation.

    Using the example give, a whole laptop, not just the battery, is designed for minimum price, low weight, competitive specs, and a relatively short life cycle. This means charging and discharging the battery far more deeply than you would for optimal service life - having a battery that's twice as heavy as a competitors while delivering the same discharge time is not a wise tradeoff in the consumer space, and nothing near a 40 year lifetime is required/beneficial. You might also cheap out on charging circuitry, and you're stuck putting the battery somewhere that can get rather warm at times.

    Contrast that to something like the mentioned satellite installations - service life and total cost of ownership are big factors, and it's very much worth using a larger, heavier battery, treating it carefully, and never fully charge/discharge it. The installation is engineered for a much larger service lifetime, at the expense of other tradeoffs.
     
    LETTERBOY likes this.

Share This Page