Clicky

Case Study: All Web Pages Load Slooooowly

By Kary | case study

Aug 03

So far I’ve mostly shown you concepts and techniques to analyze performance. Well, now I’m gonna solve a real world problem with packet analysis and walk you through it how I did it.

Brandon joined the PacketBomb email list and we were email chatting about packet analysis. He mentioned a problem he was trying to solve: web pages were loading very slowly. He sent me a packet capture, I looked it over, sent my feedback, and he solved the issue the next day. Brandon got to fix a really painful problem and was the hero at work that day. Check out the video to see how it went down.

If you want to join the Packet A(nalysis)-Team, subscribe to the email list below. I’d be glad to help you solve a painful problem or help guide you through your own analysis.

Share this post! Spread the packet gospel!

Facebooktwittergoogle_plusredditlinkedinmail
Follow

About the Author

I like being the hero. Being able to drop a bucket of root cause analysis on a burning network problem has made me a hero (to some people) and it feels real good, y’all. Get good at packet analysis and be the hero too. I also like french fries.

Leave a Comment:

(8) comments

BigPapaGotti August 4, 2014

Very good informational video. I’m glad you walked us through your troubleshooting process as I learned a lot from this video. I hope to see more videos like this with real world issues and solutions.

As an FYI there is a typo on your page. Just below the video where you say “It you want to join the…..” I think you meant to type “If you want to join the…..” :-)

Reply
    Kary August 4, 2014

    Thanks, BigPapa. I’m glad you find it useful. And, ugh, I hate typos. Fixed. Thanks.

    Reply
Kishan Pandey August 4, 2014

Excellent video kary,I remember once hansang said that for to be a good packet troubleshooter you should read RFC’s and do packet capture for everything if possible.

Reply
    Kary August 4, 2014

    Thanks, Kishan.

    Hansang, as usual, has good advice. RFCs are how it’s “supposed” to work. Packet captures are how it’s actually working (or not working). To be able to spot the issues, you need to understand how it should work vs how it’s actually working. Though keep in mind that something may not be RFC compliant but still be how it’s designed to work for that product.

    Reply
chrismarget August 4, 2014

Nice work again, Kary.

I came up against a problem much like this once. My company installed a private line to a customer site. The other end of the private line had HTTP servers. Clients couldn’t display the web pages.

The TCP 3-way handshake went quickly, much like in your captures. The client sent his GET, and the server’s TCP ACKed it.

…Then… Nothing. For 75 seconds.

Finally the server delivered an empty segment with FIN, and the connection closed gracefully.

75 seconds was my clue to the problem. I knew from Stevens Vol II that many TCPs will spend no more than 75 seconds in SYN-SENT state. You can prove this to yourself by typing ‘time telnet 1.1.1.1’ at the command line in OSX.

HTTP requests (and the TCP flows containing them) from client systems were being snarfed up somewhere and responded to NOT by the real webserver, but by a transparent proxy server near the Internet edge of the network. The problem was that the proxy server in the Internet edge didn’t know about/couldn’t reach services in the partner extranet.

So, the transparent proxy grabbed the connection, pretended to be the webserver then silently closed the connection without comment when it couldn’t reach the real service.

Man-in-the-middleboxes like this are evil.

Reply
Avi August 5, 2014

Great video.
Thanks for taking the time to record it!!!

Reply
Helmuth February 24, 2015

Great approach to the problem. Amazing with no other infos than the capture file how fast you figured out the cause.

regards
Helmuth

Reply
Wim Marien September 19, 2015

Nice video

However, there is one more thing in this packet capture whih caught my attention when watching your video. I do see there are protocol version mismatches. In HTTP, this could possibly also cause issues.
The client is requesting a webpage using HTTP/1.1, but the server (probably the content filter) is replying with HTTP/1.0. Probably it would be better to have it use HTTP/1.1 also

Reply
Add Your Reply

Leave a Comment: