But no matter how well a laboratory engine test is presumed to simulate actual driving conditions – and enormous amounts of effort are dedicated to ensuring it does – its ultimate validity is how well the engine test data correlates with oil performance data from vehicles driven in actual day-to-day service, in fleet or field tests. Field testing has been declining for many years, however, in part because these tests are very expensive and time consuming. They must be very carefully designed and conducted so that any data generated meets rigorous scrutiny.

Field tests have also lost ground as the lifespan of new engine oil categories has been significantly shortened over the past decade to meet the accelerating pace of technological change. Today, a field test may take too long to complete to be useful for evaluating oils for a new category, or to validate a new engine test.

Another drawback to much of the field testing thats undertaken: The data generated is usually not publicly available – although it does add to the information base. Oil and additive companies use road tests to fine-tune their products. And a representative of one major U.S. OEM recently told LubesnGreases, We conduct a lot of tests for our own purposes, and we use these test results both in developing our own company specifications and recommendations but also, and just as important, in how we interpret and apply this data during the development of industrywide specifications and limits.

So field testing hasnt died. Most original equipment manufacturers, in fact, mandate that suppliers of both factory- and service-fill oils be prepared to conduct field tests. DaimlerChrysler, for example, advises, Fleet testing under conditions defined by DaimlerChrysler Corporation may be required to confirm product performance.

Further, the American Petroleum Institutes engine oil licensing guidelines caution, It is the responsibility of any individual or organization introducing a new [oil or additive] technology … to ensure their engine test results still correlate with customer field service.

Getting more from less?

Yes, as an industry weve probably reduced the overall amount of field testing, confirms Lew Williams of Lubrizol. But what we have done is become much smarter about the testing weve done. The number of vehicles has dropped, but the test design has been greatly improved so that the data we get is much more useful than before.

Steve Marty concurs. Hes director of field testing at Southwest Research Institute in San Antonio, the worlds largest independent lubricant testing laboratory. In the past, a remote field test may have consisted of putting the different test lubricants in a vehicle then showing up six months later, taking the vehicles apart and seeing how things look, he said, whereas today the operational and performance data can be downloaded while the vehicle is in service via a cell phone in real time. Final parts review is still critical, but now we have more operational and performance data while the test is in process. We can instrument vehicles to record almost any parameter, and data can be collected when certain conditions are triggered. The process gives us a much more focused look at actual service operations.

He adds, The new technology helps our customers better understand what field conditions really exist out there.

As for the future, Marty foresees, As vehicles become more and more system-integrated with a variety of [electronics] controlling different vehicle operating systems, well be able to plug directly into a car and pick up a lot of the vehicles history right from the vehicle. Onboard systems are moving forward by leaps and bounds every year.

Thats still in the future. What about todays engine tests and road tests? Field tests conducted by Lubrizol and a major OEM offer a glimpse into the soundness of two engine tests: the Mack T-11 for diesel engine oils, and General Motors Sequence IIIG, developed primarily for gasoline engine oils. Testing details and data from both tests were made publicly available – one via an SAE technical paper, the other by agreement of the OEM.

Testing the T-11

The Mack T-11 test is a requirement of the recent API CI-4 PLUS heavy-duty engine oil supplement launched in the fall of 2004 and will be a requirement of the next diesel engine oil category, PC-10, now under development and scheduled for commercialization in mid-2006 (see story, page 36).

The T-11 was developed by VolvoMack at its Powertrain Division in Hagerstown, Md., under the guidance of senior staff engineer Greg Shank. The test costs around $53,000 to run and evaluates viscometrics related to the soot loading (at 6 percent) of crankcase oil for diesel engines equipped with exhaust gas recirculation (EGR). The turbocharged test engine has six inline cylinders and runs for 252 hours at 1,800 rpm. It uses about 14.75 gallons of fuel per hour – more than 3,700 gallons total – containing 479 ppm of sulfur.

Lubrizols field test consisted of one 2003 Mack CFV713 Granite dump truck with an AI-427 engine. The truck was loaded with 56,000 lbs. of crushed limestone and driven 1,000 miles a day between Cleveland and Marietta, Ohio – three trips a day at 340 miles round-trip. The recommended oil drain interval was 250 operating hours or 10,000 miles. The test was considered ended when the vehicle had completed 15,000 miles, the soot had exceeded 8 percent, or the kinematic viscosity had risen to greater than 50 cSt at 100 degrees C.

Eight different SAE 15W-40 oils were tested in this single vehicle, following identical test parameters for each.

Charts 1, 2 and 3 illustrate the results. Chart 1, Soot Accumulation Rates, shows the very similar soot profiles for five field trials compared to a typical soot profile for a T-11 test. Chart 2, T-11 and Field Test Success shows a test and field success using a CI-4 VolvoMack licensed EO-N Premium Plus oil. And Chart 3, T-11 and Field Test Failure shows a test and field failure using a reference oil with known borderline properties.

To date 15 test runs have been completed on this vehicle, the final test report summarized. This field test work illustrates that the T-11 engine test is predictive of field performance for soot-related viscosity increase. Remaining data not reported in this report support the conclusions of the study but is considered proprietary.

Scrutiny for the IIIG

The ASTM Sequence IIIG test, sponsored by General Motors and co-developed with Southwest Research Institute, PerkinElmer Automotive Research and OH Technologies (a hardware-parts supplier and development company), measures valvetrain wear, viscosity increase and high-temperature piston skirt deposit formation. Its a requirement of GF-4, the new gasoline engine oil specification issued by vehicle manufacturers Jan. 14, 2004, and of API SM oils.

This second field test was presented to SAE in late 2003, just before GF-4 debuted. This collaborative field test with DaimlerChrysler used four taxicabs in Las Vegas to evaluate sludge, varnish and viscosity increase. Two cabs were lubricated with an ILSAC GF-3 quality, SAE 5W-30 oil; the other two with a SAE 5W-30 meeting GF-4. Test miles ranged between 26,000 and 34,000 miles.

The test showed that average sludge and varnish deposits for the GF-4 oil were very favorable and significantly better than for the GF-3 oil (see Table 1). Raters gave the GF-4 oil a 9.4 on engine varnish, for example, a much better score than the GF-3 oils 5.7. Sequence IIIG results show dramatically greater viscosity increase with the GF-3 oil compared with the GF-4 oil, and a clearly better result for the GF-4 in weighted piston deposits, too.

Chart 4, Field Data – Used Oil shows the dramatically lower viscosity increase for the GF-4 oil (the yellow line) compared with the GF-3 oil at 3,000, 6,000 and additional miles of service.

The Last Word

At Decembers meeting of ASTM Committee D-02, Petroleum Products and Lubricants, GMs Bob Olree looked ahead to GF-5 and asked, How much traditional field correlation for all these new tests is really needed? and answered, A strict insistence on rigorous field correlation is cost prohibitive today. Engineering judgment must play an increasing role in test development. He went on to add, Weve got 50 years of experience which we can use responsibly.

So has engineering judgment picked up the slack where field testing has faltered? For the Mack T-11 and IIIG, that seems to be the case – so far.