Regulations Specs & Testing

Why Are Engine Tests So Difficult to Develop?


Why Are Engine Tests So Difficult to Develop?
© TAMARA; Marek; Dmitriy

Casual industry observers, and even insiders, may feel frustrated when engine oil specifications are delayed—sometimes by years. The problem tends to be worse for light-duty specification upgrades than for their heavy-duty counterparts.

The industry struggled to piece together the latest passenger car engine oil specification, ILSAC GF-6, dragging out its debut to May 1, 2020—nearly four years past its original due date. The delay became problematic enough that a supplemental category was implemented in May 2018, called API SN Plus, to help protect engines against the known problem of low-speed pre-ignition.

Such delays aren’t exclusive to North America’s specifications. The European Automobile Manufacturers Association’s new ACEA oil sequence was supposed to be issued by the end of the 2018 but has been pushed to the second half of this year. In August, it too announced a revision to its current specification to address immediate concerns.

“ACEA are looking to include the Sequence IVB, Sequence X, Toyota Turbo deposit, Mercedes-Benz M271Evo and Volkswagen TDI 3 engine tests into their updated sequences for ACEA 2020,” said Philip Reeve, director of United Kingdom-based ADLU Consultancy. “ACEA are still targeting the fourth quarter of 2020 for the update. However, industry meetings have ground to a halt as businesses have been disrupted by the global pandemic. As not all test developments have been completed, meeting the fourth quarter timing whilst including all new tests appears very challenging.”

While debates concerning specifications and test limits can take time, they are normally settled relatively quickly once the tests are established. Rather, the cause of every major delay in sources’ memories stems from development of acceptable engine tests. 

If it is critical to meeting automakers’ desired performance, just one delayed test can set back an entire category. The more new tests included in a specification, the longer the delays. For example, ILSAC GF-2 needed a new fuel economy test, and that specification was implemented on schedule. But ILSAC GF-3 saw a delay of about two years when every test was upgraded and the industry moved from leaded to unleaded test fuel. ILSAC GF-6 needed to replace all but the Sequence VIII bearing corrosion and viscosity stability test, leading to a four-year delay.

For many stakeholders, these blown deadlines are completely unacceptable. For OEMs, tardy categories can slow the uptake of increased emissions system protections, fuel economy improvements and hardware protection. This was the case for new, smaller turbo-charged engines suffering LSPI damage as the new Sequence X test was hashed out.

For additive companies, delays impact development and resources for the new technology. Engine oil marketers are unable to predict how and when the awaited category will impact the marketing and supply of their new products. 

New additive and base stock technology may require capital investments that can take years to plan and construct. No one wants to invest in facilities that may not be needed for years after they are built.

Robert Stockwell, Chevron Oronite’s OEM and industry liaison for North America, has spent his career working on engine test development. “There are two common reasons for engine tests,” he explained. “The first is to fix a problem that already exists. Developing a test around a known problem and an identifiable operational cycle is relatively straightforward and can often be completed in a couple of years.

“The second is to prevent anticipated future problems. This is much harder because new things come to light during test development and can become more important than initially thought. In many cases, the industry develops new tests that mimic earlier tests but for newer hardware,” he added. “This is a relatively straightforward process that can be completed in a few years if the newer hardware produces a similar degradation mechanism as the old hardware. But complications can and do arise with this type of approach.”

Adding to the Challenge

Over the years, engine hardware and technology have evolved, carving out more pieces to fit into an already challenging jigsaw puzzle. Advances in instrumentation and automation make it more difficult to develop tests that can properly discriminate between an acceptable engine oil and a failing one. 

Gordon Farnsworth, a retired research associate engineer with Infineum USA, worked with OEMs and test labs to develop new engine tests. “Engines have gotten a lot smarter since I first got involved in test development,” he said. “Early on, engine controls were fixed, so you could be sure you were testing the oil and not the engine. Today, engines have learned to adapt to driving conditions, so one of the first things you need to do is to work with the OEMs to deactivate some controls.” 

Mike Lochte, director of the engine lubricant research department at Southwest Research Institute in San Antonio, Texas, added, “The ECM [engine control module] will adapt engine operating parameters such as spark timing, fuel injection timing and duration, exhaust gas recirculation and other parameters to optimize efficiency and minimize emissions.

A piston is rated after an engine test at Southwest Research Institute. © SwRI

“For fuels and lubricants testing, we desire to make the next test look just like the last test, so the OEM that sponsors the test will typically lock out some of the adaptation of the ECM. Tests have also been developed with fixed valve actuators to replace ECM-controlled variable valve actuators. All of this work takes time and coordination between multiple groups,” Lochte said. 

Angela Willis of Willis Advanced Consulting and formerly of General Motors, added that ECM software can interfere with engine dynamometers. “Sometimes the actual engine calibration needs to be modified for the test behavior itself,” she added. All of this work is costly for OEMs.

Sometimes a change in engine hardware can slow things down. For example, the Sequence IVA cam lobe wear test was run in a Nissan KA24E engine with a slider-design cam follower bearing system. The Sequence IVB is run in a Toyota 2NR-FE engine with a roller follower. The roller-follower design reduced wear so much that it became difficult to test oils for wear protection. As a result, the Sequence IVB was the last test completed for the long-delayed ILSAC GF-6.

“Newer engine components are very durable, thanks to newer designs and improved manufacturing practice. This makes it much more difficult to create a wear test with a modern engine because most engines are very robust, and the amount of wear experienced is so minor it is hard to measure.”
– Mike Lochte, Southwest Research Institute

“The amount of wear on the cam was so small, the industry group had to focus on very sophisticated measurements of the wear of the valve lifter,” which the camshaft presses against, said Lochte. In addition to average intake lifter wear, a maximum for end-of-test iron loss was added as a fail-safe to look for wear over the entire engine.

“Newer engine components are very durable, thanks to newer designs and improved manufacturing practices,” he explained. “This makes it much more difficult to create a wear test with a modern engine because most engines are very robust, and the amount of wear experienced is so minor it is hard to measure.”

The problem also crops up for heavy-duty engine oils. “A few years ago a batch of piston rings for the Mack T-12 test was made with an incorrect coating. The parts were so durable they could not be used to measure wear, and the batch could not be used for ASTM testing,” Lochte said.

“Today’s cars are also less prone to forming sludge due to changes in operating temperatures and blow-by [gas] rates,” added Farnsworth. “Fuel can play a critical role in the formation of sludge, and blending the right test fuel for the Sequence V [deposit] test has become one of the industry’s greatest challenges.”

But technology is also improving the engine tests themselves, said Farnsworth. “In earlier days, lab technicians would check parameters manually, maybe once an hour. Computers now can monitor measurements every second and control things in a very tight range for a wide swath of lubricants and viscosity grades.” While this improves precision, he pointed out that such controls drive up the cost and time to develop an engine test.

A Peek Behind the Curtain

Stockwell has another theory for why engine test development seems to take so long these days: “Historically, test developers often ran the tests in-house for a year or more before the oil industry ever saw them. Many issues were discovered and resolved during this early, in-house development. In contrast, the recent Sequence IVB test was developed for the most part in plain sight, and everyone got to see all the challenges.”

Sequence IVB engine test stands at independent lab Intertek in San Antonio, Texas. © Intertek

Cost is another factor, according to Lochte. “Each passenger car engine test could cost the company that volunteers to run it tens of thousands of dollars in parts, labor and fuel. In the case of a diesel engine oil test, the cost could be way over $100,000 for each test. It is a hard decision to commit that much money to something that might or might not work, not to mention the risk that somebody might vote down the idea later on.

“Aside from the technical challenges,­ there are usually different companies that are working on it simultaneously in different labs, which slows down the process. The collaborative nature does help build consensus, though, as for ASTM test methods to become reality, they have to pass an ASTM ballot.” 

Willis agreed. “Getting the reproducibility across the different labs while at the same time accommodating each lab’s unique footprint is a challenge. It is one thing to get the proper differentiation or test behavior on one stand in one lab. But taking that and scaling it up is another story.”

The oils themselves have changed, as well. ILSAC GF-6 introduced SAE 0W-16, the thinnest oil to be included in the specification, but formulators and additive companies are testing products as thin as SAE 0W-8, a viscosity grade that some Japanese OEMs are already recommending. The same engine tests may also see much thicker oils, such as SAE 15W-40, and are required to produce reliable results at each extreme.

Will It Ever Improve?

Significant efforts in the industry to help replace old tests and advance new tests appear to have stalled. While new specifications won’t be needed for several years, some industry veterans are concerned about the longevity of testing equipment that will be used not only for API and ILSAC approvals, but also for ACEA sequences and the Japanese Automotive Standards Organization’s new low-viscosity GLV-1 category. 

The next heavy-duty engine oil category, PC-12, may well need some new tests, too. Some OEMs have stopped producing the hardware used in some tests, and older engines may not be representative of newer hardware that newer engine oils must protect. 

“If the industry continues to lean on engine tests for engine oil specifications, then I foresee this to only become more difficult, especially with OEM resources shifting to electrification technologies.”
– Angela Willis, Willis Advanced Consulting

Stockwell urged the industry to start test development earlier, including securing the needed resources. “Unfortunately, this has proven extremely difficult and is not likely to change without a major shift in how the industry cooperates on test development,” he lamented.

“If the industry continues to lean on engine tests for engine oil specifications, then I foresee this to only become more difficult, especially with OEM resources shifting to electrification technologies,” said Willis.   

Steve Haffner is president of SGH Consulting LLC. He has over 40 years of experience in the chemical industry, primarily with Exxon Chemicals Paramins and Infineum USA. Contact him at or 908-672-8012.