A little past the end of February I started having problems with my internet connected devices. In the basement we have a Roku box that the wife uses to watch Netflix. She reported that it was no longer connecting to Netflix. We’d had issues before with it needing to be re-registered with Netflix, but that did not seem to be the case. I’d click on the Netflix channel and it would say “retrieving movies” for a while and then pop back to the main menu. At first I thought something was wrong with the Roku box, so I tried the Amazon channel, but that worked and I was able to watch my content. I figured it’d resolve itself. So she just popped in the latest DVD from Netflix into our DVD player. Later that night she was in the bedroom and learned that our Samsung BluRay player was no longer connecting to Netflix. I thought that was weird, but figured maybe it was a Netflix problem. I checked on my computer and I couldn’t log into the Netflix site. Neither could Danielle on her computer. These were Linux boxes (Fedora and Ubuntu respectively) so I tried on my Windows computer. Strangely, that one could log in. That’s weird. I tried on both Firefox and Chrome with no difference. So then I tried the guest computer – that computer hadn’t been used since December and I knew it was working for Netflix back then. That would help me eliminate the possibility that I’d installed a distro update that had killed it for me. (I knew that didn’t totally make sense because of the BluRay Player and Roku) That one could reach it either. What was going on here? Was Netflix blocking Linux? Well, I figured it might go away so I waited until the next day.
The next day my wife realized that the American Express site was working strangely on her computer. I noticed that flickr wasn’t working correctly. In fact, a bunch of sites that required signing in and HTTPS were not working correctly. So I fired up wireshark on my Linux computer and I saw this:
So basically, it was chocking on one packet that it kept asking to be sent over and over and over again. What was going on here? And why was this allowing the Windows computer to work? Well, I know that the Windows network stack is not the same as the Linux network stack, so maybe that had something to do with it. What to do now? I don’t make any configuration changes to my modem/router. I’ve set static IP addresses in all of the computers individually rather than on the modem. But, just in case, I checked every page of every setting on the modem. I couldn’t find anything that was different than before or that would be causing this. I checked each setting at least 2 times and some of them I checked three or more times. I tried what you’re always supposed to do first if there’s an internet problem: I unplugged the modem/router for a few minutes and plugged it back in. I was still having issues. I asked on identi.ca to see if anyone else was having problems connecting to Netflix via Linux. No one else was having issues. I went on Freenode.net’s #Fedora room and someone there worked with me for a few hours, but we couldn’t see anything I was doing wrong. He wished me luck. Kopete started acting really strangely because it was having issues connecting to my accounts and it started locking up the computer via sending a non-stop bunch of notifications to kde4-notify. So I had to refrain from using it during the following two weeks.
I wasn’t having any luck. So I decided to tweet about it. In the past, when I had issues with Comcast, just tweeting with hashtag #Comcast would get their attention and I’d get help. In fact, it’s come to the situation nowadays where most companies won’t try and help you until you start tweeting about how things are going wrong. So I sent out a bunch of tweets tagged with #Verizon and #FIOS. I got no response. Well, even though flickr and Amex weren’t working correctly, I decided to call Netflix. Maybe a bunch of websites had just implemented a new version of TLS that was breaking Linux or something. After I told him everything I’d done – he was honest with me and told me that if I’d done all that – it was what he was going to have me do anyway, so he couldn’t help me. I thanked him and he wished me luck.
So I posted to LinuxQuestions.org. I figured they could help me if this was really affecting other people with Linux. You can read all that and see how unhelpful it was. I was very grateful that people were trying to help me, but if you look you can see that people weren’t reading what I wrote and were coming up with nonsensical solutions that ignore the way the web works. So, since that wasn’t going anywhere, I decided to finally turn to Verizon. Anyone who’s as technical as me knows that 8/10 times, these calls to tech support are pointless. The people they have working there have to go by a script. And 99% of the time they don’t know anything except for what’s in the script. And the 1% of the time they do know, they still need to go through the script first because their calls are being monitored. I got one of the former people. I started off with doing it online. The guy chatted with me and had me reset the modem and type ipconfig /dnsflush and a bunch of other things. None of that fixed it. After an hour or so, I got disconnected from him. He never called me back despite getting a number from me for which to call me back. So, an hour later, I called tech support. They also went through the same stupid things – not listening to me about my wireshark or https. Why? Because they don’t have a clue what I’m talking about. It was just some tech support dungeon in India or Bangladesh. But even if it was in the USA, again, most of the time these guys don’t know anything other than what’s in the script. He told me he’d put in a support ticket with the technical guys and he’d send me an email with info about the ticket. He sent me an email, but it was just full of links to the online help on Verizon.
So I was stuck again. But I preferred to think that he did have someone working on it and that he didn’t lie to me. All this time I kept tweeting, hoping that someone would read my tweets and try and help me out. No dice. I was losing my faith in social media to be helpful. Almost exactly a week after my devices had stopped working, I decided that maybe if I posted in the Verizon forums, someone from Verizon tech support would see it and see what was going on. So I posted here. I made a nice, long detailed post of everything I’d tried so I wouldn’t get another “reset your modem” reply. In fact, here’s what I wrote:
Starting around 21 Feb a bunch of sites started working really funky. On Netflix I cannot login. On flickr I can’t post to group forums and other little functionality is broken. I can’t sign into Facebook. I can’t sign into Verizon.com sites (more on how I’m writing this in a second) The rest of the non-interactive internet works perfectly fine so it’s not a DNS issue.
So, there is one computer on which everything is working fine (that’s how I got here to post) – otherwise I’d just think it was a network issue. All the rest of my four computers do not work. Also, my BluRay player and Roku box no longer can connect to Netflix.
Since one computer works fine you might be thinking – “oh, he changed something on those others”. Actually, nope. I even tested on a computer I hadn’t used since December and that was working back then, but now cannot access those same sites. I even booted into a live CD of Fedora Linux – so it’s completely clean, DHCP – can’t log into Netflix. Also, I’ve tried both Firefox and Chrome – both of which work on the computer that is OK.
I ran wireshark on one of the affected computers – basically it’s dropping packets on TLS sites. Just over and over and over again – all I see is TCP Retransmission. Obviously can’t run wireshark on my BluRay player or Roku
First I checked all my settings, rebooted the computers, etc. Eventually, I called tech support. Well, first I chatted with tech support. He had me run ipconfig /flushdns and he had me reset my modem – including pushing the little button on the back with a pen. Confirmed that everything reset because I lost all my wireless settings. After the modem reboot I lost him (as I expected would happen, despite him telling me it wouldn’t). Then I got on phone tech support. Again, they had me check DNS and crap like that. I know it can’t be that stuff because how would I even do ipconfig /flushdns on my bluray player?
On my own, I’ve tried using Google’s DNS servers at 22.214.171.124 as well as using the two nameservers I see in my router’s webpage. (The one at 192.168.1.1)
So what could it possibly be? A broken router? I assume if something’d changed at the neighbrhood level, they’d be getting tech suport calls up the wazoo, so it’s probably just me? I’m really annoyed because of the inconsistency – why does one computer work and none of my other devices? The PS3 was on the fritz around the same time (kept complaining about DNS issues), but it “got better”.
I doubt it’s something I’ve configured on my router since I rebooted it and cleared the settings. The working computer is on static IP address, but so is every other device here. Also, the liveCD of Linux was DHCP. I know people like to come on forums and swear down and up that they haven’t changed anything and then it turns out they did a system upgrade or changed something they thought was insignificant – but I literally have touched nothing. I didn’t even realize at first because the wife is the heavy user of the Roku and BluRay player. And she knows well enough not to touch anything because she hates interruption in service. (And I double-checked the settings anyway) ANYWAY – the 2 months untouched computer and the liveCD Linux are proof that it’s nothing that’s changed.
Anyway, it’s been a week now since everything broke. And I when I got off the phone with tech support they said they put in some kind of ticket for me. I can’t find that on the Verizon website to check the status.
So I was hoping I could get this problem solved somehow by posting on here. The guys you get when you call tech support are clearly following a script and have no clue what’s going on. I’ve loved FiOS ever since I moved into my new house (couldn’t get it in my apartment) and I really hate Comcast, but as of right now, my Internet connection is pretty useless. I can’t use my Netflix streaming devices (save the one PS3), and the reason we have more than one computer is so that we can do stuff no matter what computer we’re on. So if I want to manage my Netflix queue, I shouldn’t have to go to the one computer in the house that can get to Netflix.
Here’s hoping for some help! Thanks!
Unfortunately, it seems that the forums are just for end users. FORTUNATELY, it helped me find out that there were a bunch of others with the same problem. In fact, as you read through the pages you can see that it was affecting Mac users AND Linux users. And other people’s Roku boxes and BluRay players. Good, so I knew it was something I didn’t do! OK, this seemed huge. A bunch of people in Maryland with non-MS OSes couldn’t connect to HTTPS sites with FIOS. Perfect! Fine, if Verizon wouldn’t listen to my tweets, perhaps it would listen to the news media! That’s another thing I’d seen. People using twitter to get the media’s attention and getting relief from the big companies. So I tried to contact Farhad Manjoo, the Slate/NPR tech correspondent. Unfortunately, he has thousands of followers so I didn’t make it in his radar. Well, one night when I was having some insomnia, I went to check the Verizon forums and saw that Verizon had a twitter page. So I went to that page and saw that they have a twitter account for support and people seemed to be getting help on there. So I followed them and tweeted to them about my problem. They gave me a link for tech support. I couldn’t find that link anywhere on the site before, but maybe I was looking somewhere else and was blinded to it. Once I got there it was great. The next day they investigated and found out that there were a bunch of us in Maryland with the same issue. That Friday morning I got a call from their network guys saying they were working on the issue and it should be resolved by the end of the day and, indeed it was.
So what conclusions can we draw from this? There are too many people on twitter now. You can’t just complain to the world and hope to get restitution unless you’re a celeb with thousands of people to retweet for you. So if you want to use social media to get help, look for the twitter account for the company you have beef with. Make sure you’re thorough before you contact them so you don’t waste a bunch of time going through the easy stuff. Or call their tech support to have your butt covered if you think you might miss something easy. Even techies like myself have felt sheepish because the “non-working” device was not plugged in. And post to as many relevant forums as you can so you can help figure out if the problem is local or global.