Why Do My Results Differ From Speedtest.net / Ookla Speed Tests?

[CA3LE]

The most common question that people ask us is why their results here differ from speedtest.net. I want to explain something that may help you understand why this is.

According to the Ookla Wiki [updated link] the following is true about your tests taken at speedtest.net...

• The fastest 10% and slowest 30% of your results are DISCARDED
• The remaining data is averaged together to determine the final result

Throwing away the fastest 10% and slowest 30% of the results in my opinion does not make for an accurate test. Isn't THAT the data that you're here to see?

Also note this isn't only true for speedtest.net. Ookla is huge, most of the speed tests online run their software.

Source: wiki.ookla.com

Multithreading makes your connection look better
Ookla speed tests are also multithreaded. Meaning that they open more than one connection to the host and combine the speeds. This often can mask congestion issues. For instance, imagine that you have a pipe along your route that's limiting you to 10 Mbps. If you open more than one connection through that pipe you'll be able to achieve a faster speed... but testing that way will not clue you in that there is actually a problem along the route. Remember, you shouldn't have to multithread your connection to pull your full speed.  

 

Multithreaded speed tests, like the majority out there, are designed to benefit the Internet provider not the consumer.  They often display your maximum throughput not your throughput over the course of an entire upload or download.  Omitting the worst portion of your test resulting in inflated scores that may make you feel warm and fuzzy but aren't going to help you see and resolve connection issues.
 
Multithreading can show that you're able to max out your connection by combining the speed of concurrent connections but a great connection can max out without having to multithread.


So, if you have to multithread your connection to get your full speed, you should be asking why that is.  With TestMy.net it's instantly apparent if there is a congested route. Your speed here reflects the actual loading time of data within your browser... not the combined efforts of multiple threads being altered (dropping the top 10% and bottom 30% of the results) and added together, all through a plugin that isn't suited for the task.

 

:: EDIT ::

If you'd like to multithread with TMN, I offer that as well.  Try the multithread speed test.  TMN's multithread test gives you the option to select from an array of servers.  Allowing you to test across multiple routes, to more than one server at a time.  TestMy.net is the only Internet speed test with this ability.

So THAT'S why your results differ.   --  TestMy.net is a harder test to ace and a lot less forgiving than other speed tests.  But isn't that what a benchmark should be?

:: EDIT ::
Another thing to consider is if you're running windows, particularly if it's older than Windows 7 you may need to tune your TCP stack to see your full speeds. Windows doesn't always come out of the box optimized for fast connections. Flash based speed tests fail to detect this problem.

To make this change for free I recommend TCP Optimizer. This makes changing those settings very easy and nearly fool proof. Just open it, slide the bar over to your speed that you're supposed to have, check "modify all adapters" ...apply the settings and reboot. You should have faster speeds after your re-test if that was the problem.

Here's an outside source talking about exactly this issue, I'll hunt down more examples. >> Download Speed Test - Something fishy?

A few related topics ::

• Accuracy..
• Slow upload, drastically different from other speed tests
• Resuts Vs. Actual Download speeds?
• TestMy.net shows different results than other speed tests
• TestMy v Ookla
• Difference Between Speed Test Sites?
• Satellite Service Testing
• Locking at around 14Mbps
• Questioning speedtest.net
• resolved with TestMy.net

 

Outside Sources ::

• Internet providers caught inflating speed test results [myce.com]
• Use Testmy.net or else!! NO MORE speedtest.net posts will be allowed!!! by the Admin on [xplornetsucks.com]
• Accurate Speeds Or Speed Test Error? on [wildblueworld.com]

[mudmanc4]

Great explanation .

At times I'll use there service for a server as close to me as possible , so i can isolate the issue a bit more , true enough , thier closest server generally gives me very close results to yours here at tmn , imagine that.

edit: And that is a 1200 mile difference, let alone a non flash based test.

[TriRan]

i usually see very different results than here, when my internet is having issues speedtest.net doesn't usually detect them at all it will show my full 50/5 where as here i can see my service is really suffering and only receiving half that

i actually get better results from here then i see on speedtest.net maybe that's just me...

[CA3LE]

i usually see very different results than here, when my internet is having issues speedtest.net doesn't usually detect them at all it will show my full 50/5 where as here i can see my service is really suffering and only receiving half that

i actually get better results from here then i see on speedtest.net maybe that's just me...

Thanks for the testimonial

[dn0]

I also have the 50/5 service. But I regularly see 50-120mbps down from other flash driven sites, but I know that is false. Wish I could pull 100mbps+, but that is just a dream.

Thanks for all your hard work CA3LE!

[CA3LE]

I also have the 50/5 service. But I regularly see 50-120mbps down from other flash driven sites, but I know that is false. Wish I could pull 100mbps+, but that is just a dream.

Thanks for all your hard work CA3LE!

Thanks for the testimonial... keep visiting!

[tdawnaz]

my results there are always out of this world...great numbers...like vanity sizing...but if i really wanna know if these jeans make my butt big tmn is the looking glass i believe...not always the numbers i wanna see...but hey sometimes the truth hurts...

like right now i'm having an issue that began in early april...most likely my modem...testing here toldl me something is wrong...i called cox and they wanted me to do a test using speedtest inside their network...i said i wouldn't go by that (first off i don't go to any websites or dl anything off their network that i know of)..i told her .that i could already tell her that what it was gonna say...25+...here lately i'm getting 8 or so...so just for the hell of it i took a test there...yep...34...if i went by them i'd never know i have a problem at all...so yes these jeans right now do make my butt look big...but they lied to me

i love tmn

[CA3LE]

my results there are always out of this world...great numbers...like vanity sizing...but if i really wanna know if these jeans make my butt big tmn is the looking glass i believe...not always the numbers i wanna see...but hey sometimes the truth hurts...

like right now i'm having an issue that began in early april...most likely my modem...testing here toldl me something is wrong...i called cox and they wanted me to do a test using speedtest inside their network...i said i wouldn't go by that (first off i don't go to any websites or dl anything off their network that i know of)..i told her .that i could already tell her that what it was gonna say...25+...here lately i'm getting 8 or so...so just for the hell of it i took a test there...yep...34...if i went by them i'd never know i have a problem at all...so yes these jeans right now do make my butt look big...but they lied to me

i love tmn

Very nice testimonial... your charts clearly display where the problem happened. Looks to me like... about April 13th... Funny thing is, your upload sped up at the same time your download decreased.

https://testmy.net/stats/?&t=u&x=60&l=250&q=tdawnaz

[xs1]

I use many speed test sites. Testmy.net is by far the most " real time " results you are going to find. Other flash based speedtest sites , aka speedtest,net show your absolute best connection speed with a huge overhead buffer. That is considering NOONE around you is using the internet and you are the ONLY one online that day on your node lol.. If you want accurate, right now speed test results, i use testmy.net. I want to know my connection speeds in real life, not what they possibly can be in absolutely optimal conditions ( again due to the overhead) .

[TJS]

... condensed

and yet you have a 70 mbs results from testmy.net when im pretty sure i saw that your on a 50 mb connection?

[CA3LE]

and yet you have a 70 mbs results from testmy.net when im pretty sure i saw that your on a 50 mb connection?

Good point... but you failed to factor in that I run this site and I'm purposely trying to break things as I develop. That score is no doubt inaccurate... but it was scored that way durring trials... most likely that score wasn't even logged by the public site that you all see

Here's what I scored on a combined test a few seconds ago... on the public end

... I also have scores that are only a few kB/s in the database --- never trust my scores out of the database, unless I post a single score... It's part of my development process.

And for comparison here's what I just got on a test moments later at speedtest.net, closest server. (15 minutes later)

And my server is like 1400 miles away from me.

[xs1]

Good point... but you failed to factor in that I run this site and I'm purposely trying to break things as I develop. That score is no doubt inaccurate... but it was scored that way durring trials... most likely that score wasn't even logged by the public site that you all see

Here's what I scored on a combined test a few seconds ago... on the public end

... I also have scores that are only a few kB/s in the database --- never trust my scores out of the database, unless I post a single score... It's part of my development process.

And for comparison here's what I just got on a test moments later at speedtest.net, closest server. (15 minutes later)

And my server is like 1400 miles away from me.

Alot of times you cannot choose the closest server on speedtest.net. If i choose Tampa, which is 50 miles away, or Miami, or ANY of the close servers ; i get a 1/4th of my speed. But if i choose Atlanta GA , 900 miles away, or Huston TX , or any of Texas for that matter, i get top speeds Go figure.

[tdawnaz]

and besides...our img is prettier

Edited May 9, 2011 by tdawnaz
because apparently i couldn't spell when i posted this :-D

[CA3LE]

I use many speed test sites. Testmy.net is by far the most " real time " results you are going to find. Other flash based speedtest sites , aka speedtest,net show your absolute best connection speed with a huge overhead buffer. That is considering NOONE around you is using the internet and you are the ONLY one online that day on your node lol.. If you want accurate, right now speed test results, i use testmy.net. I want to know my connection speeds in real life, not what they possibly can be in absolutely optimal conditions ( again due to the overhead) .

Thanks,

Yeah, who cares about what your line is capable of. Don't people already know that when they order their service.

I didn't originally build this site to be used by anyone except myself... it was part of my 'toolbox'. A way for me to ensure that when I left a customers house they were getting what they paid for. ... I wouldn't build or use a tool didn't serve it's purpose. (that goes for any tool) I'm not a cable installer anymore but I'm my own biggest user of the service. Many of the features the site now has came from my own need for that service. My point is... this site is a tool, not a novelty.

I'm happy and humbled that so many people have chosen my site over the years. Thanks to all the TMN supporters.

- Damon

[TriRan]

speedtest doesn't usually pull accurate numbers for me.. take now for instance i'm having pretty big local issues here modem keeps dropping out, signals are bad.

so when i test at speedtest its as though nothing at all is wrong.. look

if i go by that then i'd never know anything is wrong with my connection. but the ISPs can't fool me... i'm going to keep having them come out and look into my issue forcing them to spend thousands until they fix it and give me what i'm paying $89.99 a month for, a good company wants a happy customer and i will be one when they fix my problems which they are trying to do

if i test to testmy.net or any other speedtest site that does not use flash i see results like

as you can see speedtest isn't even close. even if i test to a server over 1400 miles away on speedtest its still showing incorrect results

[CA3LE]

so when i test at speedtest its as though nothing at all is wrong.. look

Thanks for the info, helps prove my point.

[KHC]

Okay. Weird and semi-happy results to report.

I've just swapped out my old cable modem (Motorola SB5101) for a new Motorola SB6120, which is a DOCSIS 3.0 modem. It took Comcast a few minutes to get the new model number and ID to reset it (remotely) to get me reconnected. As soon as I was up again, I tested my connection speeds. After a couple of hours, I tested my speeds again. Just right now:

I'm finally getting over 30Mbps download numbers here! How about that?! And it now matches the numbers I get at Speedtest!

The weird thing is that my upload speed has dropped to around 1.4-1.6Mbps, whereas previously I had 4.2Mbps (Speedtest) and 3.3Mbps (TestMy). I plan to call Comcast after a full day of using the Internet.

Subjectively, web browsing feels the same!

[CA3LE]

Okay. Weird and semi-happy results to report.

I've just swapped out my old cable modem (Motorola SB5101) for a new Motorola SB6120, which is a DOCSIS 3.0 modem. It took Comcast a few minutes to get the new model number and ID to reset it (remotely) to get me reconnected. As soon as I was up again, I tested my connection speeds. After a couple of hours, I tested my speeds again. Just right now:

I'm finally getting over 30Mbps download numbers here! How about that?! And it now matches the numbers I get at Speedtest!

The weird thing is that my upload speed has dropped to around 1.4-1.6Mbps, whereas previously I had 4.2Mbps (Speedtest) and 3.3Mbps (TestMy). I plan to call Comcast after a full day of using the Internet.

Subjectively, web browsing feels the same!

Nice...

... See, another example. You completely had a legitimate problem. TestMy detected it. Speedtest.net didn't.

I just hope that you help me spread the word.

... We've known this here forever but changing public opinion is hard. People like "pretty" things, apparently whether they work or not. I try to make thing look nice here but at the heart of the site functionality and accuracy is what I'm always focused on. You're here to test your internet, not be shown a seemingly random number. TestMy truly tests your internet speed.

Thanks for the testimonial. Note to everyone, please post your personal experience! I appreciate your support.

[KHC]

Nice...

... See, another example. You completely had a legitimate problem. TestMy detected it. Speedtest.net didn't.

I just hope that you help me spread the word.

... We've known this here forever but changing public opinion is hard. People like "pretty" things, apparently whether they work or not. I try to make thing look nice here but at the heart of the site functionality and accuracy is what I'm always focused on. You're here to test your internet, not be shown a seemingly random number. TestMy truly tests your internet speed.

Thanks for the testimonial. Note to everyone, please post your personal experience! I appreciate your support.

Well now it's especially weird that you read my report today as a testimonial for functionality and accuracy because as I had noted, my overall internet experience feels the same. I'm actually neither arguing for nor against the meaning of random numbers. My feedback from today's tests just show that ONE POSSIBLE or LIKELY cause of Speedtest's number not corresponding with here is the DOCSIS 2.0 versus DOCSIS 3.0 local equipment. That's all. I'm still exploring the Internet for the rest of the day and onwards, so I'm hoping to find a FIVE-FOLD speed increase somewhere, which should be the case and be so noticeable since I'm coming from a meager TestMy number of 6Mbps before the DOCSIS 3.0. So far, subjectively, IT'S THE SAME. Speedtest before was 30Mbps = very quick user experience. Speedtest now is 30Mbps = same very quick user experience.

Only thing now is that TestMy is also 30Mbps. So the numbers now jive. Whoopeee.

I should note that I *DO* appreciate your test site and these forums and this discussion. But please do not read my results backwards for your own agenda.

[CA3LE]

I should note that I *DO* appreciate your test site and these forums and this discussion. But please do not read my results backwards for your own agenda.

Not at all, but seriously... did they detect the problem? The evidence leads one to believe that your results from speedtest.net were wrong.

... You had a problem. You changed your modem and you're now testing fine. Just because you were still able to comfortably surf doesn't mean your speeds weren't being effected.

It has nothing to do with docsis 3 vs docsis 2. Your cable modem was bad. You tested here and TestMy gave you the correct result, you changed your hardware and the result is now what it should be. That seriously can't be argued with. It's hardware man... In this instance, TestMy wins. -- It's not a coincidence that I hear the same thing time and time again. I've also recently seen it with my own eyes. Bad modem, speedtest.net reports good speeds. Swap the modem out and results here go up but speedtest.net is relatively the same.

Put it this way, if your modem is dropping packets and speedtest.net removes the bottom 30% of your results they're going to report normal speeds. Because that bottom 30% of the results is where the problem is expressed!

Imagine this, say this is the flow of data...

| = good flow

_ = dropped packets

|||||||__||||_______||||||_|||_|||||||||___||||______|||||__||_

Now remove the bottom 30% of the results

||||||||||||_|||||||||||__||||||||||_|||||||||||

Looks like there's hardly a problem now. Alternatively, we don't alter our results here what-so-ever.

I appreciate your patronage by the way.

[tdawnaz]

VeryKen...the exact same thing was happening to me...my modem just nosedived in april after many years of service...MANY...my tests here showed me when it happened...and i knew it was my modem because of a decreased signal that my isp was seeing...about the same time people started asking why the difference in numbers between test sites...so i checked out a few and most showed the numbers i pay for and better...only here at this site was the problem detected. i replaced my modem with a new motorola sb6120 same as yours...and bingo back in business.

so bottom line is if people want the fluff and vanity sizing then go to the other sites...but for those that aren't afraid of the truth and really want to diagnose the problem...come here and test and post up in the forum what seems to be ailing your machine and all the free "doctors" here will help ya make her all better

[KHC]

Not at all, but seriously... did they detect the problem? The evidence leads one to believe that your results from speedtest.net were wrong.

... You had a problem. You changed your modem and you're now testing fine. Just because you were still able to comfortably surf doesn't mean your speeds weren't being effected.

It has nothing to do with docsis 3 vs docsis 2. Your cable modem was bad. You tested here and TestMy gave you the correct result, you changed your hardware and the result is now what it should be. That seriously can't be argued with. It's hardware man... In this instance, TestMy wins. -- It's not a coincidence that I hear the same thing time and time again. I've also recently seen it with my own eyes. Bad modem, speedtest.net reports good speeds. Swap the modem out and results here go up but speedtest.net is relatively the same.

Put it this way, if your modem is dropping packets and speedtest.net removes the bottom 30% of your results they're going to report normal speeds. Because that bottom 30% of the results is where the problem is expressed!

Imagine this, say this is the flow of data...

| = good flow

_ = dropped packets

|||||||__||||_______||||||_|||_|||||||||___||||______|||||__||_

Now remove the bottom 30% of the results

||||||||||||_|||||||||||__||||||||||_|||||||||||

Looks like there's hardly a problem now. Alternatively, we don't alter our results here what-so-ever.

I appreciate your patronage by the way.

Okay, here's another case in point.

You remember I mentioned that my upload speeds suddenly became just 1.4- 1.6Mbps from either SpeedTest or TestMy? (Also same from Comcast's test site). This compares to a consistent 4.3Mbps at Speedtest and 3.2Mbps at TestMy previously. So my concern became why is the upload now just about half the speed it was before the modem upgrade?

I got on the phone with Comcast Tech Support. They tried resetting the modem several times. I went through VERY CAREFUL and isolated direct computer-to-modem tests and boot-up sequences with and without my router... several times.

Now I'm back to a consistent 4.4Mbps at SpeedTest and 3.4Mbps at TestMy. These scores are again uploads.

So what does this tell you now? How does TestMy scores help better than SpeedTest scores in this case?

Again I'm not knocking your test site here.

You just seem to be jumping to the wrong conclusion. You seem to be too anxious in proclaiming that TestMy is better than SpeedTest. I'm not saying that it's not. Yes it might be true. But so far my test results and observations only show that ALL TEST SITES are helpful — NOT that TestMy scores are any MORE accurate!

TestMy.net is indeed a VERY USEFUL tool. But my testing today is NOT proof that TestMy scores are more truthful. The truth is not that easy. You cannot claim that TestMy wins from what I reported. JUmping directly to that conclusion is not using scientific logic. There are other factors involved. There are other possibilities. And like I said, I will have to keep experiencing the Internet looking for definitive cases that make it feel a whopping FIVE TIMES quicker than before. For now I'm not experiencing any real world benefits from the SB6120. Subjectively it's just the same!

For example, before (using SB5101 and TestMy @7Mbps download speed), I downloaded the iTunes 10.3 upgrade in just a few seconds. Now (using SB6120 and TestMy @30Mbps), I downloaded the iTunes 10.3.1 update also in just a few seconds! Both were, as I vaguely recall, around 70MB. Same blasting speeds!

If anybody has suggestions to try — to clearly distinguish my speeds are now "better" I would love to hear it. The core question here *MIGHT* become, "is the user experience significantly better at 30Mbps SUSTAINED compared to 30Mbps momentarily boosted?" That's just one of the other possible factors. OR, the core question might be, "would MOST internet users notice the difference between 7Mbps and 30Mbps download speed?" You see where I'm getting at?

[CA3LE]

So what does this tell you now? How does TestMy scores help better than SpeedTest scores in this case?

It doesn't... at all. Because you don't have an issue anymore. But can you trust the alternative now? You saw yourself that they reported you at full speed. ...Your speed at TestMy didn't magically change when you changed your modem. You were having an issue and you most likely weren't getting your full speed, whether you noticed it or not. The fact that your speed improved here after changing a hardware component clearly displays a difference between the two sites.

I never say that I'm better than any other speed test. I only present the facts and I let my visitors make their own decision.

p.s. I forgot to say before... my "agenda" is to improve the Internet.

[SlowFreddy]

Not much difference. I never get 1 meg up..

[TriRan]

i to do not see much difference when my connection is working correctly the problem occurs when my connection is sucking, speedtest.net will say everything is fine and this site will validate that my connection is indeed sucking