Lost 5kts - what does this data mean?

I'm running a follow up performance test and seeing about a 5kts loss. Here are the specifics: (Test1 vs Test2)

Static RPM: 2250 @ 200'DA vs 2310 @ 1000'DA

GPS box speed test =
Altitude: 7200' vs 6500'
DA: 8300' vs 8200'
WOT RPM: 2725 vs 2675
Calculated speed: 161kts vs 156kts

No configuration or system changes between the two tests, just environmental conditions.

What additional data should I gather and what areas of investigation should I pursue?

Thanks for your opinion and expertise.

Could be a lot of things - including leaning technique, OAT, aircraft weight....

Glen,

My first thought is to fly it again and see what it shows. More data is better, and other things can vary, even with similar DA. Even at the same DA, differences in altimeter setting (if significant) can make a difference in manifold pressure and HP. Since you had the same DA at a lower altitude on test 2, the OAT must've been higher, so that could have an impact as well (I know DA is DA, but was your engine running hotter, and along those lines, were you're leaning techniques similar/identical?). Humidity can have an effect too. The 50 RPM difference between tests could, and probably did, account for some of the speed delta, even if that sounds like a small change in RPM. How consistent were those RPM readings, and what type of prop do you have? (I'm assuming FP, since you don't mention MP.) If you have a MP readout, you should record that too. Were your fuel loads the same? All this stuff can be cumulative.

You mention a GPS box, so was this a 4-way test? If so, and if you used the NTPS spreadsheet from Kevin's Horton's site, how did the standard deviations compare (that will tell you about the quality of the data). Were your techniques in the test similar/identical, and did you record data at points where your altitude control was spot on? Was the air smooth or bumpy during the tests, and were the bump conditions similar/identical? I've found air "quality" and flying technique can really make a difference in results. Sometimes after a run I just know that the results will be taken with a grain of salt, due to how it "felt".

I'm not a TPS grad, so Ken, Kevin and others that are may have better gouge for ya, but IMHO, one or two flights does not make for a definitive result. Ya probably need more data to get a pattern established.

Just some thoughts...have fun testing!!

Cheers,
Bob

Test method and post test data processing are critical

5 knots is a big difference and indicates invalid testing. Tests when the wind is calm for best results but the NTPS spread sheet removes the wind effect significantly.

Try this:

Set altimeter to 29.92 climb to 6,000ft

Check outside air temperature if it is above 3C, subtract 3 and descend 100 feet X the remainder. If it is below 3C, subtract 3 and climb 100 feet X the remainder.

Set the throttle wide open, Prop max (if constant speed), mixture for best speed/power ~100 ROP.

Fly 360, trim for hands off level flight set the autopilot for the 360 ground track if you have one otherwise fly a GPS ground track (not heading) of 360. Set the altitude hold if you have one at the altitude determined earlier that will provide 6000 ft density altitude.

When the speed stabilizes make a GPS ground ground szp[eedspeed recording every 20 sec. until you have five in a row that are within one knot of each other.

Turn to 120 and repeat

Turn to 240 and repeat

Go home, average the five consistent recorded speeds on each leg.

Enter the Average GPS speeds and GPS ground track angles into the three leg worksheet option (saves about 1 gallon of fuel per test and is accurate) of the NTPS spreadsheet.

Then do it again (I actually do both test runs on the same flight) average the two results to give you confidence in the test validity) you should have your top speed at this point

Bob Axsom

Some answers / some added data

Bob (rvmills) ...

OAT/DP at Airport (50' MSL): 19/10 vs 23/19
ALT at Airport: 30.30 vs 30.20
Winds at test altitude: 12@260 vs 5@045
CGT (only have simple gauge) 325 vs 370 (hottest)
CGT (only have simple gauge) 270 vs 300 (coolest)

Things that were the same for both tests:

No MP readings. Flight experience would be characterized as "pleasant". Leaning procedure was "old school style" of lean to onset of roughness and richen slightly. Flight pattern was to fly cardinal compass headings - cross checking mag compass and DG for stable heading - and test run starts once heading is stabilized. GPS box was flown three times. Load was same for both tests (Pilot, full fuel, aprox 30lbs of misc).

Things that were the different between tests:

The weather was definitely different. The CG was different with 79.61 vs 81.36 (RV-8 with 1090 empty; the CG change was the move of tools etc from aft baggage to fwd baggage).

The RPM difference at test altitude concerns me. I just do not have sufficient experience to know what to attribute it to - weather, pilot, aircraft, etc. I would like to assume the pilot did everything correctly.


Bob (Axsom) ...

Thanks for the improved test methodology. My throttle settings are likely very close to your recommendation (within the limits that I have a FP). My altitude was not as you describe so I will use your method going forward. I did trim for hands off and set AP to hold heading and altitude. I thought heading vs track would insure better evaluation of wind but I can change to track when I repeat the test. I did record multiple GPS ground speeds, waiting for it to stabilize but not the 100 seconds you suggest. I do repeat test runs within a single flight.


Scary side note - I'm actually enjoying this type of testing.

Glen,

Things I note for run 2 (the slower run) are:
Hotter
Higher humidty
Lower ambient pressure
More FWD CG
More wind

The sum of those would likely make a difference. How much, hard to tell, but likely some. The lower CGT is an intersting result. Could be different leaning levels, given the old school method (nothing wrong with that, but probably hard to find best power with it), could be that the motor was hotter to begin with (was the first set of readings on a second hop of the day...heat soaking can make a difference too).

Lots of subtle things can make differences. More testing will add data points and the trends can be seen more clearly. Good news is that it is kinda fun!

It'll be interesting to see how follow on tests play out. Have fun!

Cheers,
Bob

The sample intervals are 20 sec not 100 & NTPS req. GPS Track

humptybump said:

Bob (rvmills) ...

OAT/DP at Airport (50' MSL): 19/10 vs 23/19
ALT at Airport: 30.30 vs 30.20
Winds at test altitude: 12@260 vs 5@045
CGT (only have simple gauge) 325 vs 370 (hottest)
CGT (only have simple gauge) 270 vs 300 (coolest)

Things that were the same for both tests:

No MP readings. Flight experience would be characterized as "pleasant". Leaning procedure was "old school style" of lean to onset of roughness and richen slightly. Flight pattern was to fly cardinal compass headings - cross checking mag compass and DG for stable heading - and test run starts once heading is stabilized. GPS box was flown three times. Load was same for both tests (Pilot, full fuel, aprox 30lbs of misc).

Things that were the different between tests:

The weather was definitely different. The CG was different with 79.61 vs 81.36 (RV-8 with 1090 empty; the CG change was the move of tools etc from aft baggage to fwd baggage).

The RPM difference at test altitude concerns me. I just do not have sufficient experience to know what to attribute it to - weather, pilot, aircraft, etc. I would like to assume the pilot did everything correctly.


Bob (Axsom) ...

Thanks for the improved test methodology. My throttle settings are likely very close to your recommendation (within the limits that I have a FP). My altitude was not as you describe so I will use your method going forward. I did trim for hands off and set AP to hold heading and altitude. I thought heading vs track would insure better evaluation of wind but I can change to track when I repeat the test. I did record multiple GPS ground speeds, waiting for it to stabilize but not the 100 seconds you suggest. I do repeat test runs within a single flight.


Scary side note - I'm actually enjoying this type of testing.

Click to expand...

The sample intervals are 20 sec not 100 & NTPS spread sheet requires GPS Track.

Bob Axsom

Thanks Bob, I did not realize the NTPS calcualations assumed GPS. Thanks for that tidbit. As for my "100 seconds" was I doing "math in my head" and referring to your recommendation of 5 readings, 20 seconds apart that are all within 1kts; thus consistency for a 100 seconds window. I was probably only looking at a 45-60 seconds window. It's a case of "more is better".

I can tell three things from my current data - (1) I need a more formal report and data gathering along with more parameters; (2) I need more reports before I can declare a baseline or a trend; (3) I will be watching that initial report of a 5kts loss.

I wish I had more historical data to help determine if the 5kts is all or in part attributed to air frame or power plant issues.

Scotty, I need more data !

Thanks to Bob and Bob, I have a more formal test procedure. It is already proofing valuable. I'll come back to this thread in about a week, once I've used the new procedure on different days and different times of day to report the findings.

One subtlety I missed in the recomendations was about "the location for test runs". I'm located on a narrow stip of land with water on either side. In a series of test runs late yesterday, I noticed speed fluctuations over different areas. I'm guessing thermal activity. The most stable set of numbers I recorded were when the entire test procedure was over water. I'll be validating this theory per the common week.

Since I am targeting cross-country flying, I've changed the initial altitude from 6000, to 8000' and then make the temperature dependent adjustments from there. Two reasons for the change - (1) I assume as long as all tests use the same procedure, I'll have a valid procedure ; second, the new propeller that will get tested - after I have a satisfactory baseline - was designed based on 8000'DA.

Your observations are correct

I am based in northwest Arkansas at FYV (Fayetteville Drake Field) and I have the Ozarks all around me. Most of my test runs (I have done hundreds of them) are to the east under a MOA at 6,000 density altitude.

When I first put my clean piece of paper on my knee board I write the date and what what I am testing at the top. When I get the ATIS I record the altimeter setting and the information identifier there as well just for reference because depending on what altitude I need to fly the test at, I need to have a method to tell Razorback Approach the altitude I will be flying at in an altitude reference that they understand.

On my 360 leg I have a reasonably level piece of land to fly over, east of Springdale and Rogers. If the speed stabilizes before rogers it is best - it is much easier to meet the 5 consecutive 20 second interval reading consistency requirement. If not the list of recordings can get quite long before 5 consecutive 20 second interval readings are within 1 kt of each other. When I turn to 120 I will initially have a relatively consistent surface but if I don't get my 5 early I will be flying over mountains and the readings will get less stable. When I turn to 240 there is no efficient way to avoid flying over mountains. When I complete the 240 leg I boundary line my data sheet for test one, turn to 360 and repeat the process. That is the best test method I have been able to come up with to date.

Your observation of difference based on where you are flying is correct but thermals are only part of the inconsistency problem. Changing wind velocity, updrafts and down drafts also affect the GPS ground speed which the NTPS spreadsheet calls for. You are smart enough to understand that so I will not waste any time on explanation. A way to minimize the effect is by doing your testing early in the morning or (less favored) late evening on days when the wind is calm. I have mods to complete and races to go to so I just have to go when I can and try to get good data - its is not perfect.

Because the conditions are not static, even though you fly your tests back to back they will not be exactly the same on two consecutive runs but if you control every thing else correctly they should be within 2 knots of each other. I used to just make one run for time and money economy but now I just bite the bullet and make two. John Huft told me once that he uses the 4 leg method and makes 3 or four runs to refine his absolute performance numbers. I don't think that is necessary to get relative performance numbers to test a mod which is what all of my testing is for.

It is important to establish you airplanes density altitude baseline for testing so that the test results are relevant to one another - once you select 8000 do every single test at 8,000 ft density altitude or the results are not directly comparable.

After I enter my leg speed average numbers into the NTPS spreadsheet and I get my speed and wind numbers I print the sheet or sheets for multiple runs and I staple them to the in flight raw data collection sheet. I write on every sheet the date, what was being tested and I sign them before filing for later reference.

Bob Axsom

Bob, I really do appreciate you sharing your experience and expertise in comparative testing. Trust me when I say I am paying attention !

FYI - probably should have chosen a better title for this thread: something like - "performance tests of Aymar Demuth vs Catto Prop" - but interested readers will find it anyway.

Test Platform is my 150hp slow build RV-8. It's not the cleanest or lightest RV-8 but it's mine

Tests using a modified version of the methodology described by Box Axsom where I chose an 8,000' reference altitude. FYI: It turns out that simply substituting 8000 for 6000 from Bob's method actually results in a normalized test altitude of 8300 ft.

All tests were WOT and leaned aprox 100 ROP

The propellers being tested are the original 2-blade Aymar Demuth 68-71 all wood propeller with epoxy pain finish and the Catto composite wood core 2-blade with nickel leading edges.

After 6 runs with the Aymar Demuth propeller and 6 runs with the Catto propeller the average was 157.8 kts vs 161.4 kts. If I throw out the high and low from both groups, the numbers do not change much with 157.6 kts vs 161.4 kts.

The Catto was manufactured as a direct swap for the AD. This resulted in a significant gap at the backer plate as the AD propeller was much rounder than the Catto. Filling this gap may contribute to slightly better performance of the Catto but that was not part of the test.


A few interesting details ...

Craig had not previously built a 2-blade propeller for a RV-8 with only a 150hp lycoming. Based on conversations, he predicted a 2750 rpm at WOT and a 4kts speed increase. The WOT max RPM during all tests was 2750. Averaging across all tests, there is approximately 3.75 kts speed increase.

The weight is 11 lbs vs 12.5 lbs
Static is 2300 rpm vs 2025 rpm
WOT is 2725 rpm vs 2750 rpm

CHTs are slightly lower with the Catto - likely attributed to the more refined airfoil approaching the hub.
Climb is about the same.
Fuel burn at cruise RPM is about the same.

While a little extra speed is nice, the primary reason for the propeller testing was the nickel leading edges. However, if you can get an Aymar Demuth propeller that matches your engine, its a very good performer for the price.

I chatted with Craig regarding the performance numbers. He said the only changes he could make were to trade cruise for climb. The prop was maxed out.

He estimated the 3.75 kts equate to 12hp more going to thrust. I don't know propeller aerodynamics but I do know I'm going faster this the same engine.

I'm in the middle of a regional cross country. Typical flight for me - 9500ft or 11,000+ ft DA. Little to no wind advantage. Calculated TAS was 142kts and average fuel burn over the 470nm 3.6 hr flight was exactly 7gph. Obviously the climb was in the 10-11gph range and cruise decent was in the 3-5gph range.

Of course, when all the numbers are crunched, the performance increase equates to just 6 minutes.