We are in the process of migrating this forum. A new space will be available soon. We are sorry for the inconvenience.

Network issue - slow during peak hours to Verizon


Draco
2015-06-03, 11:36 AM
Just to give an update to this situation...

The TLDR version is: "It's beyond our control, but you can try contacting your ISP"

Two snippets from the tickets (I had two replies/interactions):

To conclude it seems that it is beyond our control that you are experiencing changes in the download speeds. I know it's probably hard to bargain with an ISP but you could try and see if you can possibly change route.
I can guarantee to you that we always try to get in touch with every global network provider, not only to refine and have more acute routes, thus better speed for you. Obviously, those types of negotiations take time and the outcome is hardly predictable, hence we can offer you any guarantee through that end.

Since this is again beyond our immediate control, we will keep pushing but you could also try to contact your ISP as suggested to see if you will get a faster result.
Since I cannot talk directly to a tier 1 global provider (Tata Communications or Telia.net) as a customer of an ISP (they require circuit id's and the likes), I begun a process with escalating up the chain at my ISP, Time Warner Cable. I'm currently awaiting a call back from the level 3 department, AKA the department that filters/legitimizes requests then creates a ticket to TWC's NOC/GNOC. If the conversation goes well today, I may have a ticket open to TWC's NOC. I don't expect much to happen after this though, but I'll be sure to update my currently-closed OVH ticket with TWC's ticket number. I additionally will contact OVH by phone afterwards to discuss. (See my FYI below)

Maybe after this creation, both OVH and TWC can work together to help better the connectivity between providers. (If this gets the route fixed, it's possible this may only solve my issue with TWC, but won't do anything for Verizon customers depending on how it's fixed.)

FYI: Yesterday I experienced crippling speeds as low as 300 Kbps (OVH > TWC on the download), the worst I've ever seen, which seriously impacted the usage of my server for anything.

----

Got the call, played along with directly connecting to the modem *rolls eyes* and submitted the proper info for them to escalate to TWC's NOC. FYI: At 9-10PM the speeds were abysmal yet again (1-5 Mbps) and the route was still the same. I will await response from TWC before contacting OVH again. (Unless OVH would like me to call them up? I doubt it.)

Draco
2015-05-27, 09:42 PM
Thanks for the heads up Phil. I submitted the ticket.

Here's hoping we can find a root cause. Let me know if there's anything more that can be done on my (or should I say, our) end.

Phil @ OVH
2015-05-27, 07:09 PM
Hello,

Yes, there is definitely a problem on our end at the moment. It appears our main fibre-optic link to our BHS datacenter has been severed. We already have a team working on the issue, but for the moment, there is no exact ETA for the repair.

Please understand that we are doing everything in our power to resolve this issue as quickly as possible. We ask that you be patient, we will have this resolved as soon as humanly possible. Once everything is back up and running, you'll be able to submit your support ticket.

For more information, please visit our status page: http://status.ovh.com/?do=details&id...6c4f44ad0b0855

Regards,
Phil C

Draco
2015-05-27, 06:49 PM
Quote Originally Posted by Phil @ OVH
Hello everyone,

What I would recommend doing is to open a support ticket through your control panel and provide the following information:

-Results of a ping on your server.
-Results of a traceroute (preferably done with MTR) from your local host to your server.
-Results of a traceroute (preferably done with MTR) from your server to your local host.
-Results of the following command: iperf -c proof.ovh.ca -P 5 -r

Include some details on the issue you're facing, and our technicians will be able to look into and resolve this issue for you promptly!

Thanks for choosing OVH,
Phil C.
Done and done. MTR and iperf results have been generated. Once at 2:30 PM ET and at 7:00 PM ET where the issue seems to arise. (It can happen any time after 5PM)

...aaaaaand, well, this is embarrassing. This is probably terrible timing in light of your recommendation but... your control panel/API/ticketing services seem to be getting slammed or are broken, since it's loading dog-slow and won't submit the ticket properly or load the "My Tickets" area. I'll check back with that to submit it later. Maybe it's maintenance on your end? ~_~


Quote Originally Posted by Keelan
Did you ever get this figured out, i have just began to notice this in the last few days which is rendering my servers unusable, OVH is also running be around with these MTR traces. Ive contract my ISP and they are looking into it (engineering). Its only at peak times, i used to be fine but hte last few weeks i will get kb/s from the dl from OVH ul is great still.

WHat did you do?
Unfortunately I wasn't able to contact that individual but the issue still remains. If it continues after this freshly created ticket though, I'll definitely reach out.

Also, as a reference to you, here's what my iperf looked like with appropriate info removed. (it's just fine locally to OVH's iperf test server, FYI) This is testing from my Kimsufi/OVH server to my local linux instance at home. (30/5 TWC connection) This should be getting 30+ Mbps, instead it's achieving under 5 Mbps.

Code:
iperf -c home-ip -P 5 -r
	------------------------------------------------------------
	Server listening on TCP port 5001
	TCP window size: 85.3 KByte (default)
	------------------------------------------------------------
	------------------------------------------------------------
	Client connecting to home-ip, TCP port 5001
	TCP window size: 45.0 KByte (default)
	------------------------------------------------------------
	[  7] local my-ovh-server port 36195 connected with home-ip port 5001
	[  6] local my-ovh-server port 36193 connected with home-ip port 5001
	[  3] local my-ovh-server port 36194 connected with home-ip port 5001
	[  4] local my-ovh-server port 36192 connected with home-ip port 5001
	[  9] local my-ovh-server port 36196 connected with home-ip port 5001
	[ ID] Interval       Transfer     Bandwidth
	[  6]  0.0-10.2 sec  5.62 MBytes  4.63 Mbits/sec
	[  3]  0.0-10.2 sec  5.50 MBytes  4.52 Mbits/sec
	[  9]  0.0-10.3 sec  5.50 MBytes  4.49 Mbits/sec
	[  8] local my-ovh-server port 5001 connected with home-ip port 53163
	[  4]  0.0-10.3 sec  4.75 MBytes  3.87 Mbits/sec
	[  3] local my-ovh-server port 5001 connected with home-ip port 53164
	[  6] local my-ovh-server port 5001 connected with home-ip port 53165
	[  9] local my-ovh-server port 5001 connected with home-ip port 53161
	[ 10] local my-ovh-server port 5001 connected with home-ip port 53162
	[  7]  0.0-10.4 sec  7.50 MBytes  6.06 Mbits/sec
	[SUM]  0.0-10.4 sec  28.9 MBytes  23.3 Mbits/sec
	[  9]  0.0-11.3 sec  1.50 MBytes  1.11 Mbits/sec
	[ 10]  0.0-11.7 sec  1.62 MBytes  1.16 Mbits/sec
	[  6]  0.0-12.0 sec  1.75 MBytes  1.22 Mbits/sec
	[  3]  0.0-12.1 sec  2.00 MBytes  1.38 Mbits/sec
	[  8]  0.0-12.4 sec  1.50 MBytes  1.01 Mbits/sec
	[SUM]  0.0-12.4 sec  8.38 MBytes  5.65 Mbits/sec
	
	Result: Abysmal speeds. Inconsistent throughput, with wide fluctuations in speed. Felt in terminal while typing as well.

Phil @ OVH
2015-05-26, 09:40 AM
Hello everyone,

What I would recommend doing is to open a support ticket through your control panel and provide the following information:

-Results of a ping on your server.
-Results of a traceroute (preferably done with MTR) from your local host to your server.
-Results of a traceroute (preferably done with MTR) from your server to your local host.
-Results of the following command: iperf -c proof.ovh.ca -P 5 -r

Include some details on the issue you're facing, and our technicians will be able to look into and resolve this issue for you promptly!

Thanks for choosing OVH,
Phil C.

Keelan
2015-05-23, 10:43 PM
Quote Originally Posted by Draco
Hmm... that's not good. Either way, thanks for seeing the reply and giving a heads up and history on it. Pretty disappointing and frustrating to hear it lingered for so long. I had an older BHS dedicated server on OVH before this Kimsufi one and it didn't have this problem at all. :\

I'll test this out a little further over the next day or two. Since I hit up their World Tour event in NYC somewhat recently, I'll probably give a quick shout out to one of the OVH contacts I met and direct them to this thread/issue. Can't say it'll make a difference (or that you might care, since it sounds like you've fully switched) but it's worth a go.
Did you ever get this figured out, i have just began to notice this in the last few days which is rendering my servers unusable, OVH is also running be around with these MTR traces. Ive contract my ISP and they are looking into it (engineering). Its only at peak times, i used to be fine but hte last few weeks i will get kb/s from the dl from OVH ul is great still.

WHat did you do?

Draco
2015-01-16, 11:29 PM
Quote Originally Posted by mzaur8
This issue has been going on since August. I gave up and got a server with Chmuranet. No more slowdown. Oh and they actually have customer service and respond to emails unlike Kimsufi/OVH. The admins of Chmuranet will switch you to a different backbone if you have bad peering. But I'm having good speeds with level3. Anyway don't expect this issue to be fixed anytime soon.
Hmm... that's not good. Either way, thanks for seeing the reply and giving a heads up and history on it. Pretty disappointing and frustrating to hear it lingered for so long. I had an older BHS dedicated server on OVH before this Kimsufi one and it didn't have this problem at all. :\

I'll test this out a little further over the next day or two. Since I hit up their World Tour event in NYC somewhat recently, I'll probably give a quick shout out to one of the OVH contacts I met and direct them to this thread/issue. Can't say it'll make a difference (or that you might care, since it sounds like you've fully switched) but it's worth a go.

mzaur8
2015-01-16, 10:19 PM
This issue has been going on since August. I gave up and got a server with Chmuranet. No more slowdown. Oh and they actually have customer service and respond to emails unlike Kimsufi/OVH. The admins of Chmuranet will switch you to a different backbone if you have bad peering. But I'm having good speeds with level3. Anyway don't expect this issue to be fixed anytime soon.

Draco
2015-01-16, 10:15 PM
I will second this post/issue and I'm also in the BHS1 datacenter.

The only difference with my post is these are speeds to a Time Warner Cable connection in NY.

Throughput speeds *WILDLY* fluctuate between 1 Mbps to 15 Mbps (been going on for at least 2 hours now that I've noticed... sometime around 8PM ET?)

Code:
                                                   Packets               Pings
 Host                                            Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 142.4.210.253                                13.0%   376   10.6  84.5   0.4 3322. 305.5
 2. bhs-g1-6k.qc.ca                               0.0%   376    1.5   1.1   0.6   3.7   0.4
 3. mtl-2-6k.qc.ca                               16.8%   376    2.9  12.6   1.6 208.8  33.2
 4. 198.27.73.172                                97.3%   376   69.9  26.3  17.8  69.9  18.2
 5. ???
 6. if-22-2.tcore2.CT8-Chicago.as6453.net         0.0%   376   20.1  18.5  18.0  27.1   0.9
 7. 64.86.79.98                                   0.0%   376   24.7  25.1  24.4  79.3   3.9
 8. 107.14.17.193                                 0.0%   376   31.2  29.3  27.2  32.8   1.0
 9. ae-4-0.cr0.nyc20.tbone.rr.com                 0.0%   376   28.6  29.2  27.0  32.3   1.0
10. 66.109.6.77                                   0.0%   376   30.2  29.7  27.6  32.9   1.0
11. 24.164.164.255                                0.0%   376   30.8  30.4  30.0  32.3   0.2
12. agg1.sctwny0102h.nyc.rr.com                   0.0%   376   43.1  47.4  31.9 122.1  10.6
-----snip----- :)
I have confirmed full throughput speeds (11MB/s) to other servers on other providers that are not TWC-related. So it appears the path to Verizon and TWC is having problems on the OVH -> TWC/Verizon path only. Uploading throughput to OVH/Kimsufi servers remains speedy/fine.

I'm hoping we can be advised of what might be going on here.

mzaur8
2014-12-28, 06:24 PM
Hello,

I have a BHS server which is very fast to my primary location in the US (Nashville - Comcast). I'm currently in my second location in New Jersey where there is Verizon and I noticed the speed gets very slow. During peak hours the download speed from the server can get slower than 2mbit. Right now it's about 3mbit (7pm EST Sunday). I remember a month ago I noticed this but I saw there were some tickets at OVH which showed capacity issues between BHS and NWK, but they have since been resolved. Currently there are no open tickets showing issues in the OVH Network so I'm not sure what to do. But I'm still having the issue where every evening the speed gets slow coming from the server (otherwise the Verizon connection is fast). It is only my download speed that is affected. I can upload to the server at my max speed of 50mbit while only getting 3mbit down. The issue usually resolves after 3 hours but it's consistent every night.

Here is a traceroute from the server to my modem. Probably not very helpful

HOST: ns508696.ip-198-100-145.net Loss% Snt Last Avg Best Wrst StDev
1.|-- 198.100.145.253 0.0% 10 0.6 58.3 0.5 188.3 76.7
2.|-- bhs-g1-6k.qc.ca 0.0% 10 0.9 1.0 0.6 2.3 0.3
3.|-- mtl-2-6k.qc.ca 0.0% 10 1.9 1.9 1.7 2.0 0.0
4.|-- 198.27.73.180 50.0% 10 18.0 21.5 17.9 35.5 7.8
5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
6.|-- 64.86.78.30 0.0% 10 29.4 36.3 25.6 90.6 19.7
7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
8.|-- pool-108-53-227-17.nwrknj 0.0% 10 46.8 48.5 44.4 51.0 2.1

Is there anyway to pass this info on to OVH engineers? I can run more tests to help figure out where the issue is