5 Reasons Why Smartphone "Speed Tests" Are Fundamentally Flawed

Share
Embed
  • Published on Sep 19, 2018
  • I often hear that so-called "Speed Tests" are a better way to judge the performance of a smartphone, because they are more "real life" or "real world." However, the basic ideas behind Speed Tests are unsound. Here are 5 reasons why these tests are fundamentally flawed.
    Introduction to Android app development: www.dgitacademy.com
    Let Me Explain T-shirt: teespring.com/gary-explains-let-me-explain-2
    Twitter: twitter.com/garyexplains
    Instagram: instagram.com/garyexplains/
    #garyexplains

Comments • 1 685

  • Joe K
    Joe K 13 hours ago

    I like you. You give me a reason to hold onto my phone for another couple of years. Considering I've owned my phone for now three years it's running just fine compared to other phones

  • Abdulrahman AM
    Abdulrahman AM Day ago +1

    Thank you Gary for explaining this to us!

  • Mīćhæł Pīņəß

    People: go watch other speed tests you are invalid
    Gary: tf man hell naw you are wrong
    Gary: *creates speedtest G*

  • Alvan Yap
    Alvan Yap 28 days ago

    I think yourself did the same speed test on s10 snapdragon vs exynos, so it's flaw too?

    • Gary Explains
      Gary Explains  28 days ago

      +Alvan Yap Exactly. Speed Test G is designed to fix all the flaws described in this video. I thought that was obvious. Have you watched the introduction to Speed Test G video?

    • Alvan Yap
      Alvan Yap 28 days ago

      +Gary Explains ya you use an app call "speedtest g" instead

    • Gary Explains
      Gary Explains  28 days ago

      I have never done such a speed test in my life, not on an S10 or any other device. Maybe you are confusing me with someone else?

  • Speak out and remove all doubt!

    Point 5 - Yes I agree again, its not a great way to simulate real world as that is not how we use it but as with Point 4 if you did retry to use the app for a little while the tests would get evermore unrepeatable and apples to pears.
    The one point i hugely agree with you on is analysing the actual performance in the app (at the same settings) really needs to be tested properly and we don't see this anywhere i don't believe. We do get people playing games and saying they don't notice and dropped frames or it still feels responsive but this is hugely subjective.
    My main issue with benchmarks is firstly i have seen too many times companies cheat at them either cutting corners or going into special benchmark modes or even paying (sorry 'sponsoring') the benchmarking company and then all of a sudden they are much faster. My second issue with them is we often don't see or understand what they are doing to give us these scores and how that equates to the real world. Ok the GPU benchmarks are a little easier but if they have a bios to a certain type of shader technique or particle effect or in game physics etc and you mainly play games that aren't using those then if a phone got a low score you might thing that phone isn't any good for you but actually its just not great at that benchmark.

  • Speak out and remove all doubt!

    Point 4 - Repeatability. Tests should be done with all apps closed, phone restarted, etc etc. They should also be done several times and an average value taken. I agree its never going to be perfect but again it's the closest we have to simulating the real world use of phones and the stopwatch is there as a guide to show you possibly with this phone you will be waiting a little longer than with this one.

  • Speak out and remove all doubt!

    Point 3 - Totally agree, although the only caveat to that is if you are comparing a phone with 4GB RAM to one with 12GB then it is a good way to show the benefits of more RAM, not an accurate way but a good way.

  • Speak out and remove all doubt!

    Point 2 - I agree iOS vs Android vs Windows vs XXXX will all be very different program code underneath but they are still taking you to your account of that program which will be broadly be the same, ultimately this is what the user wants to know. If i'm on Instagram and i want to open into FB which phone will be the fastest? With today's phones all being so fast its low on my priorities but when so phones are much slower its nice to know before you buy them.

  • Speak out and remove all doubt!

    Point 1 - When phones move to better and faster memory then the IO transfer speeds improve, as you said this is what is being tested in those benchmarks. So when we move to UFS 3.0 and LPDDR5 this year or 2020 then the IO speed will really improve.
    But i totally agree with you in that if they are loading different setting of the game then its pointless and unfair and a full analysis of frame rates during the game with the same settings is very much needed.

  • Ckhomphzxs Sarona
    Ckhomphzxs Sarona Month ago

    Thank you garry, NOW I KNOW👍👍👍

  • NaderZico
    NaderZico Month ago

    This video is incredibly funny and truthful

  • Sudipto Das
    Sudipto Das Month ago +1

    press F to pay respect to Gary Sims

  • superxorn
    superxorn Month ago

    Synthetic benchmarks also only tells you so much. You can have a good spec phone set up quite badly which makes it underperform in a real world scenario.

    • Gary Explains
      Gary Explains  Month ago

      Well, if a device has bloat and memory problems that are slowing it down then that would be picked up by benchmarks and even more so by my Speed Test G test. As for Apple products having slower specs, that is nonsense, Apple products tend to have the bests specs, except for RAM but that Apple has some tech which makes the RAM less important when you are just using one app at a time (but it can be a problem for split screen etc).

    • superxorn
      superxorn Month ago

      +Gary Explains I mean bad implementation by the manufacturer - full of bloat, odd memory management configuration etc. An apple product would be a spec underperformer in many cases, but it'll be zippy in use because of good software optimization.

    • Gary Explains
      Gary Explains  Month ago

      So you are saying that if a person has a badly configured phone then it won't work well in real life. OK, yes, I agree. But isn't that obvious?

  • iliyas Abdul Kader
    iliyas Abdul Kader Month ago

    Hai i need that speed test g apk
    Anybody have?

  • TillTheLightTakesUs
    TillTheLightTakesUs Month ago +1

    Thank fuck somebody said it at last.

  • Mix Eyepatch
    Mix Eyepatch Month ago

    Talking about benchmark test .... Different softwares gives different results
    Some results seem paid to give more score

  • Zachary Sturgeon
    Zachary Sturgeon Month ago

    Then why do newer phones consistently beat older phones on these speed tests if they aren't objectively testing overall performance?

    • Gary Explains
      Gary Explains  Month ago +2

      If you increase CPU performance, memory performance, internal storage performance, and system bandwidth then apps will start faster. But starting a game and playing a game are two totally different things. For example, the GPU isn't even used during the start up but heavily used during actual game play.

  • rrobertt13
    rrobertt13 Month ago +1

    THANK YOU! The internet is full with shit "tests" like this.

  • masoud omer
    masoud omer Month ago +1

    Yes finally

  • Ahnaf Khan
    Ahnaf Khan Month ago +1

    Genius

  • Rex Storm
    Rex Storm Month ago +2

    You just earned 1 sub and a lot of respect...
    Finally a 'real' tech enthusiast with actual knowledge of chipsets and software/firmware

  • rahul singh
    rahul singh Month ago

    U means RU-clipr from India are Fooling us and others

    • Anas Khalil
      Anas Khalil Month ago

      He's talking about EverythingApplePro, Phonebuff, XEETECHCARE.. Etc.
      These are some of the most famous ones, Every small youtuber tries to copy them.

  • Sourav Goswami
    Sourav Goswami 2 months ago

    If you do compare iphone with an Android, keep in mind that even the programming language is different! From lower level perspective, the kernel is different.

  • dooms cr
    dooms cr 2 months ago +1

    Finally a tech youtuber i can watch in peace. All the others tech reviewers doing the same shit over and over again.

  • Yusuf Imran
    Yusuf Imran 2 months ago +1

    Great insights as always. Gary really explains 😇

  • ירין בן אבו
    ירין בן אבו 2 months ago

    Hi Gary
    Im sorry but "attack youtubers” its wrong. Your explanation just show the people the true behind the scene. Some youtubers arent objective with their videos and misslead the public to pay a lot of money what's made them eventually disappointed with their choices [Sometime].

  • Anas Khalil
    Anas Khalil 2 months ago +1

    This is my favorite video on youtube

  • Alex K
    Alex K 2 months ago

    What would you recommend for an accurate speed test method?

    • Gary Explains
      Gary Explains  2 months ago +1

      LOL, try this ru-clip.net/video/0dsQiw-G8O8/video.html

  • Tech Pegasus
    Tech Pegasus 3 months ago +1

    Gary do a speed test g between mediatek helio p70 and snapdragon 660

  • Sundar Balaji
    Sundar Balaji 3 months ago

    Wowww, very well explained with proper analogy Gary. It's simply awesome

  • sextionmirror
    sextionmirror 3 months ago

    Gary, Can you explain about touch latency between ios and android please .I want to know .

  • Ian S.
    Ian S. 3 months ago

    I don't care about this "not science" speech, all I know and care is that I'm using my apps way faster on my iPhone than in any other android phone.

  • sextionmirror
    sextionmirror 3 months ago

    Your video should be get more view and destroy wrong logics about many test of youtubers.You preserve more subscribers.

  • Alex S
    Alex S 3 months ago +2

    What's the point of owning a fast smartphone if you don't watch this video with it?

  • Bigby Wolf
    Bigby Wolf 3 months ago

    Loved it? Enjoyed it? I WATCHED EVERY SECOND OF IT!
    Speed tests are "app opening test" im shooting this everywhere!

  • Bigby Wolf
    Bigby Wolf 3 months ago

    I AM SO SO GLAD THAT SOMEONE IS FINALLY TALKING ABOUT THIS. EXPLAINING THIS IN THE COMMENTS JUST OPENS UP HUGE GATES TO CRITICISM AND HATE

  • Kobby Sintim - Arthur
    Kobby Sintim - Arthur 4 months ago

    I love this! Very legit points. Great job Gary!

  • Sahil Mehta
    Sahil Mehta 4 months ago +2

    Phonebuff should probably delete his channel now ..

  • zed piecce
    zed piecce 4 months ago

    I normal go for phone because it's can run my prefer emulators at 30fps minimum (ppsspp, damonps2 pro and dolphin) very well.... That why I follow snapdragon series for hardware and android for OS.....thirdly the price and battery time..... I am not concern about screen and security stuff, just emulator gaming.....otherwise I go for small Nokia phone......thank to snapdragon 845 in pocophone f1....... And android support

  • Rollie Bautista
    Rollie Bautista 4 months ago +1

    Haha that the reality very nice explanation...☝️ more power for your channel sr Gary

  • royrobsn
    royrobsn 4 months ago

    Oh my Good i was sooo long waiting for a Man how understand how a Handy Works.. Keep on going!! and by the way, the stress test you have cratet is better than any test i have ever seen on playstore, it would be verry intressting to see your test running every step for 5-10 minutes, so the throtteling factor comes realy out.. I think that could complete chance the result again.. Best greats from Austria..

  • Jinosh Philip
    Jinosh Philip 4 months ago

    Gary
    You should expand your speed test to iOS as well !

    • Gary Explains
      Gary Explains  4 months ago

      Please see the FAQ. The link is in the description.

  • Mike Fedyk
    Mike Fedyk 5 months ago

    As someone with decades of experience as a sysadmin, all I can say is duh, finally and thank you.

  • DemiImp
    DemiImp 5 months ago

    To be fair, benchmarking programs are also flawed.
    Just not nearly as flawed as the app launching speed tests.

  • walter park
    walter park 5 months ago

    Terrific!
    Of course you should support meaningful tests and disillusion people about empty comparisons.
    Thanks!
    This is credible education. I like it.

    • Gary Explains
      Gary Explains  5 months ago

      I guess you haven't heard of Speed Test G!!! garyexplains.com/faq

  • Adarsh Adk
    Adarsh Adk 5 months ago +1

    I never thought like like this,thanks for proving me that I was wrong , Man you are brilliant :) good luck gary !

  • Abhishek Asthana
    Abhishek Asthana 5 months ago

    you pedantic ass, it doesn't matter. ultimately what I care about it how fast Twitter and RU-clip load on my phone. the tests are perfect for me. go lecture in a university to CS majors. they probably care about this under the hood nonsense

  • Jean Grullón
    Jean Grullón 5 months ago

    Benchmarks are less related to day to day feel than app loading time comparisons are scientific.

  • Dom Esp
    Dom Esp 5 months ago

    That means Apple kicks ass!!

  • azrul nizam
    azrul nizam 5 months ago

    Tq Mr. Professor.

  • Jarkko Niemi
    Jarkko Niemi 5 months ago

    Sounds like that the most accurate test would be something like that:
    * We use program what simulates real phone use, and it works on IOS and Android phones.
    * It does virtual touches to screen.
    * We script one day of phone-use.
    * Any phone moving must be done on automated way, for example navigation app use, by putting phone to Drone, and Drone has scripted path how it goes.
    * We repeat test ... lets say 99 times. Then we use Average and/or Main for results.

    • Gary Explains
      Gary Explains  5 months ago +2

      Good luck turning that nightmare into a RU-clip video!!! :-)

  • Louis-Philippe Duval
    Louis-Philippe Duval 5 months ago

    If some people legitimately believe that these "speed test" are accurate, they are literally brain dead.

  • Manoj Hegde
    Manoj Hegde 5 months ago

    Gary i am very curious on Google apps that comes with every Android. that is Google play services and Google play store. i see these apps slow down phones. please through some light. and even make tutorial on how Google play service work and what happens if i stop its update process for Google play services and Google play store .. appreciate your hard work

  • Darren Holden
    Darren Holden 5 months ago

    A definite thumbs up. Excellent work.

  • karthikeyan M.V
    karthikeyan M.V 5 months ago

    Speed test is still relevant . I had phone with decent processor but it took 15 seconds to load contacts app. Then I realized something is wrong with flash memory .Then I tested with androbench. It gave around 10mbps read speed . 😥. That phone is not useable anymore

  • Hitesh
    Hitesh 6 months ago

    Hey Gary. Amazing video as usual!
    I was wondering about Substratum overlays. Does it affect performance of the device? If it does and I know it does but if you can make a video about overlays, it would be great. Thanks.

  • Kyle Alexander
    Kyle Alexander 6 months ago

    it took me a while to realise on my own how dumb speedtests are but Gary really put the nail in the coffin. Thank you for this.

  • Smart Potato
    Smart Potato 6 months ago

    True that

  • aryan ambastha
    aryan ambastha 6 months ago

    And David Rahimi makes shit ton of money doing these speedtest videos! Let's Expose him!

  • Niidea1986
    Niidea1986 6 months ago

    I unsubscribed to most mobile tech channels but I think I will resubs to this one :)

  • TripleAceAAA
    TripleAceAAA 6 months ago

    Maybe. But I notice Apple is faster for the most part. So the benchmarks are higher for Apple and I notice Apple smoother and faster. So to me it adds up

  • where is my wall
    where is my wall 6 months ago

    i like this channel ..

  • Asaur Rahman
    Asaur Rahman 6 months ago

    Gary thank you once again

  • Asaur Rahman
    Asaur Rahman 6 months ago

    And what if someone moves the application s to the external sd card and show that it's a slower loading time?🤔Terrible!

  • Asaur Rahman
    Asaur Rahman 6 months ago

    this is an eye opener

  • Rocelo Becera
    Rocelo Becera 6 months ago

    Right on point Gary! Thanks!

  • MIC MUTED
    MIC MUTED 6 months ago

    Show some framerate, frame time, temperature, battery graphs and samples of the test running

    • MIC MUTED
      MIC MUTED 6 months ago

      It would give us a better idea of the real world performance of a pine if reviewers / speed testers showed them

    • MIC MUTED
      MIC MUTED 6 months ago

      +Gary Explains just saying that if someone could make software like msi afterburner and other data collection apps for mobile. It would be nice

    • Gary Explains
      Gary Explains  6 months ago

      Are you asking me to show those? On what tests?

  • anand kulshrestha
    anand kulshrestha 6 months ago

    ah i am slow but i am happy with it.

  • maen Alnjadat
    maen Alnjadat 6 months ago

    Great explanations from really scientific professional
    My question is what is related by where we can find high quality of apps as possible?
    Which operating system loads more details and frame rate of apps like games ios or Android?
    Thank you

  • Mateusz Kirow
    Mateusz Kirow 6 months ago

    I personaly think some reason could be busted. Rebooted phone/same clear phone It couldt be tested on cellular data (from the same bts, in night/when traffic is lowest) in perspective of a couple of days. Test in the same app, the same page like "shop xyz" then "shop xyz search: phone X" etc. It should give us comparative results. Mabye i'm wrong.

  • J T
    J T 6 months ago

    Look at the clips he makes. He's here to educate others, help us to understand things better.
    Others are just doing it to get clicks and subs, get richer so they can buy their 3rd house or 2nd Ferrari.
    Thats the difference. Sadly, many people out there think 'millions of subs = most reliable source'.

  • L L
    L L 6 months ago

    It's "user experience"
    Speedtests channels need to change their video title to testing ux for both phones or something like that.

  • Matthew Pool
    Matthew Pool 6 months ago

    Best pull quote: "That's not science! That's just nonsense!" Excellent video. 😁

  • Jjsm Felisilda
    Jjsm Felisilda 6 months ago

    This is just the best explanation ever... Keep it up Gary ☺

  • Aakash Jhaveri
    Aakash Jhaveri 6 months ago

    170 youtubers watched this video

  • Sam Fullbuster
    Sam Fullbuster 6 months ago +3

    OnePlus fans are so dumb I swear. Just because the phone opens an app half a second faster. I used the onplus 6 it wasnt as good as some other flagships in gameplay experience. Thanks for the video man. Hope it goes viral.

  • Anitt Bhatnagar
    Anitt Bhatnagar 6 months ago

    Nailed it.....

  • Alex Priego
    Alex Priego 6 months ago

    Great Obi-Wan....spoken you have. And amazed us you have. Thanks for the video. Always enjoy your smart professional opinions

  • Steve, TheDailyShaver
    Steve, TheDailyShaver 7 months ago

    GARY RUINS EVERYTHING ;))

  • A B
    A B 7 months ago

    At the core of it you’re not testing like for like but at the level of abstractions of the common user it’s perfectly appropriate.

  • Tim Wambari
    Tim Wambari 7 months ago

    Preach Brother! Preach! Well said :-)

  • Med Msd
    Med Msd 7 months ago

    You are right about most of it but honestly a gaming test is testing the frame rate which is a real life test with the same quality (ios and android might be different in quality)
    and for ram management I dont really care if the apps are different if the android version is better then good for android users but if you watched several videos that shows overall the ram management on oneplus 6 for instance is better then IOS or android phones then it sure gives you an idea about real life (I dont care if it is because 8gb or they had optimization) those factors you are talking about like network latency or different content are for my opinion have minimal impact and the more videos you watch the more these factors will be insignificant.

  • Atlantic Picture
    Atlantic Picture 7 months ago +1

    Thought you're going to also mention that these "tests" usually start with the idea that *a person is able to touch 2 or more devices w 2 or more fingers at identical time*

  • Khairil Ashraf Samsudin

    In Gary we trust, another great video!

  • Reymond Gopog
    Reymond Gopog 7 months ago

    This by far my favourite video of the year. Today I stopped watching "speed test" video and stopped wasting a few minutes of my life time watching nonsense speed test video. Thank you Gary. You got yourself a subscriber today.

  • Captain Storm
    Captain Storm 7 months ago

    Thanks Gary

  • john watson
    john watson 7 months ago +1

    please keep making these videos, these things are really important to know 🤟🤟🤟🤟

  • mя υηѕтσρραвιє

    Mr Garry explains you are just amazing your videos are amazing 👌👌👌

  • Dennis Deveaux
    Dennis Deveaux 7 months ago +1

    Been sharing this every time someone shares speed tests between the Pixel 3 & iPhone XS.
    There's enough little issues with the Pixel 3, but showing it in a speed test is not a fair test as you explain

  • Rajkamal S
    Rajkamal S 7 months ago

    I swear that the evidence that I shall give, shall be the truth, the whole truth and nothing but the truth. Gary Sims

  • Rajkamal S
    Rajkamal S 7 months ago

    You look a bit frustrated about this topic!

  • MIND NOVA
    MIND NOVA 7 months ago +1

    More excuses.

  • Abhishek Soni
    Abhishek Soni 7 months ago

    yes but people does't know that things

  • trevor da veiga
    trevor da veiga 7 months ago

    For all those people who think they know tech just by watching youtube videos, gray Sims has explained what performance testing is in a very kind lovely way and shows his credibility.

  • Srishti Bhagat
    Srishti Bhagat 7 months ago

    Sir

  • Lu Silk
    Lu Silk 7 months ago +1

    Enough said thanks 👨

  • Hector Rodriguez
    Hector Rodriguez 7 months ago

    I watched two of your videos and I subscribed to your channel. Very very good content and information.

  • DK Nex
    DK Nex 7 months ago +1

    Technical Legend

  • Steps Ahead IT Services

    I love the video, the only thing I don’t like is that when you are ramped up, you don’t finish the senteces with. Ya’ dumb c@nt$! Only that would make this video 200% better.

    • Gary Explains
      Gary Explains  7 months ago

      So you think that things with profanity are better... interesting...

  • David Zindr
    David Zindr 7 months ago

    This changes everything lol
    Well done great eye opener.