A Comcast voice outage yesterday illustrates the risks of network automation and virtualization and the move to the cloud.
Reports of service outages first surfaced yesterday morning, impacting business users in multiple markets nationwide, according to online news website The Verge. The report cites a statement from Comcast issued around 3:30 p.m. ET noting that the company experienced a service interruption with Comcast Business Voice and VoiceEdge Select services and that service had been restored for most customers. Comcast apologized for the outage.
Comcast Voice Outage
Comcast Business Voice Edge, launched in 2012, is a cloud-based voice and unified communications offering that eliminates the need for certain premises-based equipment and provides convenience features such as voicemail-to-email. But while the cloud-based approach enables these capabilities, it also magnifies the impact when something goes wrong. With traditional technology, outages generally were confined to relatively small geographic areas.
The situation is reminiscent of the 911 outage that affected seven states in 2014. A software issue with a cloud service used by 81 public safety answering points (PSAPs) caused all those PSAPs to become inoperative.
The FCC investigated the 911 outage and came up with six recommendations aimed at preventing future outages of that nature. While some of that advice pertains specifically to 911 service, some recommendations could make sense for a company like Comcast offering VoIP, UC or other cloud-based services.
For example, the FCC recommended that all parties involved in end-to-end call completion should practice situational awareness to better prepare for potential outages. The commission also recommended the development of a comprehensive set of best practices.
Ironically, the FCC noted that increased use of cloud-based 911 services were accompanied by a rise in “sunny day” 911 outages resulting for “purely technical reasons,” illustrating the dark side of technology advances.
It would be fruitless and even foolhardy to seek to prevent cloud-based services, virtualization and automation or to prolong the use of less efficient decades-old technology, however, as those technology advances can bring many benefits. Instead, outages such as the 2014 PSAP outages or Comcast’s voice outage yesterday remind us that the industry shouldn’t leap into the new technologies without carefully considering the downside and how it might be prevented.
Our Comcast Xfinity Business Phones just went down – again. Second day in a row. Boulder, CO.
Nashville, too!
Houston too
And Atlanta!
Chicago as well
same here. it was out yesterday and the phone recording recognized the outage. now its out again but it didnt say anything about it being out not waiting on hold for 2 hours
SWFL
Ours too! Eugene OR
Albuquerque, NM as well. Second day!!! What is going on?
Same here in Martinsburg, WV
miami down. how can i receiving the my cell phone
We are down for the second day in a row also. Washington state! Not good!
California
Plainfield IL is out again
Us too in Tewksbury MA
Same here in Charleston SC
Here in Massachusetts as well day 2 loss of business
off in Olympia WA for about an hour agin today but working now…fingers crossed
Ours did as well in Vancouver WA!!
Us too in Harrisburg, PA
Our comcast Business Phones just went down again.. Second day in a row. Lakewood WA
Our's Just came back on… Lakewood WA
Same same in Milton, WA
santa cruz, ca down yesterday 8am-3pm
Now down again today
Our Santa Cruz business lines are down again, too. We are losing a ton of business. Comcast will be losing a lot of business after this fiasco, as well.
the phones are down again in our office, Caravan Serai tours in Wallingford, Seattle. pls email us maha@caravan-serai.com or rita@caravan-serai.com
Phones down again! Boston, Ma
Same here Joliet il
Here as well… Service out again. Palm Beach County Florida
Our as well in Chamblee, GA
Phone down Wednesday and today in Chico, CA
This is all the Trump administrations fault. So much for big business being our saviors
Edwin if you are serious then you are afflicted with Trump Derangement Syndrome.
Second day in a row! Down here as well in Owings Mills, MD.
Here too in St. Augustine, FL
ours too Naugatuck CT
Second day in a row for Colorado Springs
We were down most of yesterday, and back up by 3:00 pm. Now we're down again this morning (6/7/2018).
…forgot to mention, we're in northern California.
Down in Michigan again also.
Down in Evanston (Chicago) IL. again.
Comcast you SUCK!
Here in Maryland is down too.
Down in Rockville Md Second day in a row…why no explanation from Comcast?
Down in Minneapolis, MN too (again, 2 days in a row).
Business lines down second day in a row… Cape Cod
Down for the second day in a row – Spokane WA
Miami, FL as well. Yesterday it was all day loss of phone service. Today it was working until 10 minutes ago.
And SLC, Utah. Again.
down in hillside,nj for last hour. Anybody knows how long it will go on?
We just went down too!
Berkeley, CA
cant keep on losing business
Costing millions of dollars for small business.
Out for the second day in a row.
Down in Savannah GA again
Down in the western suburbs (Chicago) – our offices are Elgin/Lombard/Oak Park – all down, second day in a row.
Yup – down again in Brunswick, Maine.
down in McMinnville Oregon also again today. we own a tow company comcast is leaving the public stranded on the road.
down in tallahassee, fl, AGAIN
OUT IN NASHVILLE TN AGAIN TODAY AS WELL
Business phones in Atlanta down again (2 days in a row).
anyone in Michigan facing trouble
Down here again for second day Nashville TN
Same in Atlanta. The problem is all phone providers are the same. So frustrating.
Indiana also..
down in NH AGAIN
second day in a row same time
WE ARE OUT IN HOLLYWOOD FL KILLING MY BUSINESS…. HELLO AT&T I WILL BE CALLING
Our business phone service is down again – Sarasota FL
Me too, Napa, CA
WINGS AUTO TEWKSBURY MA DOWN AGAIN SECOND DAY IN A ROW WHATS UP COMCAST
home health services business lines down second day, unable to transfer phone lines to cell phones, no body get a solution !!
Canton, MI 2nd Day in a row
down in Minneapolis second day in a row. Tired of losing business and looking to switch service now. This is ridiculous.
Down again for second day in a row. Northglenn co
Down for a second day in a row – Burien, Washington. This is NOT good for our business!
We are out for the second day – Tacoma, WA
Has anyone been able to contact Comcast?
Phone lines down AGAIN in Williamstown, NJ 08094
It's an hour hold time. We can't get through.
They just hang up on me
2nd day in a row for VA
Second day in a row – San Francisco, CA – Over it.
Phones down for 2 days in a row now. How are we expected to conduct business this way? – Miami, Fl
Monroe Washington down for the second day… Hard to do business like this…
Yes, out in Michigan.
Mine is down too for second day in a row and I'm in Irwin, PA
You get excited, think the lines are up, then bam, back down. Hope this is resolved soon, once and for all.
Down in Colorado Springs, CO
Same here in East Freetown, MA
Down again here. Bloomindale IL
Down second day in Savannah Ga
Still down in FL – 2nd day.
Down again here in Mableton, GA (West of Atlanta)…-2nd day in a row…Get it together Comcast. We have business to tend to…
Second Day–Bloomington, IN
DEBARY FLORIDA Second day in a row! Noon time again the phones went down!
Unacceptable!
San Carlos, Ca.. 2nd day
Business lines down second day in a row… Medway, MA
Same Here. Knoxville, TN
I wonder if they will give us a credit.
Businesses in our city down, again. Merced CA
Down in Boston
Deerfield Beach 2 days now
There seems to be ongoing problems with Comcast telephone service! As a business owner decisions have to be made if Comcast can handle business telephone service. In my opinion they are not ready for prime time. If our telephones go down we are out of business and that is the reason we stay with AT&T (20 lines still on copper. If one line goes down it is no big thing 🙂
Phones here at Stone Creek Kitchen, in Monterey, California out since yesterday (Wednesday, June 6) morning. I had to send a blast email to 4000 customers because so many coming into the store say my phone must be off the hook (yesterday callers got a busy signal). It is very peaceful here, without the phone ringing off the hook, but I know this is negatively impacting our business.
Is this poor design or cyber attack or both?