On July 20, the IO of some EVS disks in Tencent Cloud Beijing District 3 was abnormal. But I didn't expect it to be so serious, it led to the complete loss of online production data of an Internet startup's "cutting-edge CNC technology". It is not yet known whether this "complete loss of user data" is an isolated case.
The following is the communication process between the faulty user and Tencent Cloud online customer service:
The following is the user's own statement:
Frontier CNC is a start-up company that started with a WeChat public account, registered the WeChat public account "Frontier CNC Technology" in 2014, accurately positioned in the CNC, mold, machinery industry, and then the vertical industry accurately focused on more than 400,000, and in 2016 received investment from well-known companies to accelerate the development of "cutting-edge CNC technology", the company's business has also transformed from a WeChat public account to a one-stop platform for building an industry, developing products including websites, H5, mini programs, and with the world's high-end equipment manufacturing industry Germany, A number of leading enterprises such as Japan, Switzerland, and the United States have established in-depth cooperation.
In order to cope with the rapidly increasing traffic trend and the demand for security and reliability, "cutting-edge CNC technology" chose Tencent Cloud server. The disaster occurred on July 20, 2018, when nearly 10 million yuan of platform data was lost, including accurate registered users and content data accumulated after long-term promotion and diversion, which instantly destroyed a startup....
After consulting IT industry professionals, people said that it was difficult to understand: "The data is lost on Tencent Cloud because the hard disk is hung up, which is really incredible, even if a company uses its own server, the hard disk failure causes the server to go down can recover the data, this technology is not difficult, not to mention that the cloud has copies, they should have a disaster recovery mechanism and data integrity inspection to ensure that the data is recoverable; Even if all of them are damaged, as long as the hard drive is not physically destroyed, the data can be recovered..."
After the accident, Tencent Cloud initially did not shy away from the issue, admitting that they would be responsible for the accident and claiming that it was an accident they had never had before, and hoped that we would give them some time to process compensation.
But in fact, Tencent Cloud's attitude of being a large company is really unacceptable. For an Internet startup company where "time is the lifeblood", the reply they gave during the negotiation process with Tencent Cloud was always "feedback has been given to the relevant departments of the company, please be patient".
It was not until the 14th day of the accident that Tencent Cloud gave a reply, which was a devastating answer: the total amount of compensation liability will not exceed the total service fee charged by Tencent Cloud for the default service, and a package worth 100,000 yuan of Tencent Cloud (I don't know what it is). Subsequently, Tencent Cloud seemed to feel that there was something wrong with such a reply, and quickly changed the 100,000 package package in the compensation plan to 132,900 yuan in cash, saying that this was the biggest compensation they were fighting for.
It's hard to believe that this is actually the solution given by Tencent Cloud? They keep saying that they attach great importance to it, but in fact they ignore the life and death of startups!
That's the platform we've spent more than two years building! When all content data is lost, how much will it cost to get back to business in this case? Can it still be operated? What can you use this 130,000 yuan for? That's the lifeblood of our company!
We believe that the sense of justice in this society exists, and the law can protect property from loss, even if an egg touches a stone, it must be touched....
We don't know if "cutting-edge CNC technology" is a 1 in a billion chance of being damaged by Tencent Cloud, we need Tencent Cloud to give a clear explanation for the accident, give compensation that can sustain the survival of a startup, and hope that relevant righteous people will pay attention to the incident.
The following records the whole process of the accident,
July 20 (Friday)
At about 20:00, our website, mini program, and H5 suddenly appeared and could not be opened.
So I urgently arranged for a programmer to check the reason, and found that I could not log in to the cloud server, and I couldn't restart.
At 23:00 in the evening, a work order was initiated in the background of Tencent Cloud, and Tencent Cloud quickly replied: Some cloud disks in Beijing's third district failed and were being urgently restored.
July 21 (Sat)
While waiting anxiously, in the morning, our programmer did not see the reply message from Tencent Cloud. I initiated a work order again to ask whether the fault had been dealt with, but after multiple work order communication, I did not get a clear answer.
We called Tencent Cloud's emergency service call almost every half hour to urge the results, and the answer was that the data could not be recovered for the time being, and the expert team was urgently repairing.
15:49 Tencent Cloud replied, let's try to recover the data. After our programmer followed his method, he found that the downloaded data was corrupted and could not be used.
At this point, an ominous premonition ensued. In the morning, we were worried that the suspension of the product would bring a bad experience to users and bring traffic loss to the platform. But when Tencent Cloud told us that the data could not be recovered in the afternoon, we were close to collapse, and the most urgent expectation at this time was to be able to recover the data, and the loss of the platform was out of operation for a short period of time.
Subsequently, he kept calling Tencent Cloud to complain, asking that no matter what, as long as there was a glimmer of hope, the data would be restored. Tencent Cloud's reply is that the problem has been upgraded to priority and has been reported to the relevant person in charge.
July 22 (Sun)
No matter how anxious you are, it is useless, the complaint phone call is one after another, and the attitude of the customer service staff is always a standardized reply "Please be patient, the problem has been fed back".
At 12:24, I received a call from Tencent Cloud Shenzhen, a Miss Gao said that she was the person in charge of handling complaints, and she came to dock this accident, she expressed that due to the failure of Tencent Cloud's cloud disk caused data loss, Tencent Cloud has done its best, but the data cannot be retrieved, and she wants to hear the opinions on compensation.
We once again expressed that our needs are not claims, and briefly introduced the importance of data, and hope that Tencent Cloud will do its best to recover the data. Miss Gao said that there was almost no hope, she would convey our opinions to the company, make another technical effort, and promise to reply to the handling opinions on Monday.
That night, I wrote a formal complaint letter and submitted it through the Tencent Cloud background, eagerly hoping to attract the attention of Tencent Cloud's senior management, because this is not an ordinary data accident, which is related to the life and death of a startup.
July 23 (Monday)
On Monday working days, Tencent Cloud should have given opinions on how to deal with emergencies, but no!
It wasn't until 13:40 p.m. that I received a coping reply in the background of Tencent Cloud: Hello, your complaint letter and related comments have been received, we have upgraded the feedback again, if there is any progress, we will contact you as soon as possible, thank you!
Subsequently, we replied with a message: it would be understandable if the response on Saturday and Sunday was not so timely, and last night's complaint, until today's weekday, did not give a formal reply at 13:40 p.m., which is incomprehensible.
At 20:56, Tencent Yungao called to explain that 100% of the data could not be retrieved, expressing that the company had attached great importance to this accident, and the purpose of the call was to learn more about the losses caused by the loss of data in "cutting-edge CNC technology".
The whole call lasted nearly an hour, and we frankly introduced the development of the cutting-edge CNC platform to Ms. Gao in detail, and the serious consequences of this data loss. Finally, Miss Gao said that she would report in detail to the company leaders and would give us a reply tomorrow.
July 24 (Tue)
Waiting for a reply again! By 17:40 p.m., I had not seen a reply. I was anxious to go backstage to ask about the progress.
At 19:16, Tencent Yungao called and expressed that she was already pushing the company's leaders to give their opinions, and hoped that we would wait patiently.
July 25 (Wed)
After a day, waiting for Tencent Cloud's reply.
Thursday, July 26
At 15:00, Ms. Tencent Yungao called and said that she had reported our feedback, and the company boss already knew about it, but asked us to implement the claim plan in writing and send it to her by email. At that time, I added Miss Gao's WeChat.
That evening, we quickly wrote a document "Cutting-edge CNC Technology Platform Loss Estimate", including the development and related situation of the cutting-edge CNC technology product line, lost data, impact on the cutting-edge CNC technology platform, and platform loss value assessment.
In this accident, the data lost by the cutting-edge CNC technology platform includes the core data shared by PC web pages, H5, and mini programs. All accurate user data registered on the platform was lost, hundreds of thousands of user posts were lost, and industry brand database data and all entered information were lost. Because it is a highly vertical industry, it is extremely difficult to obtain traffic, and this data is gradually accumulated based on more than two years of promotion.
From 20:00 on July 20, the platform has been completely suspended, active users have gradually lost in the case of website shutdown, platform trust will drop to the freezing point, and the subsequent recovery of activity will face a great challenge at the geometric level. A fast-growing startup instantly returned to the early days of "only relying on WeChat public accounts", and the company fell into an existential crisis!
We hope that Tencent Cloud will assess the losses positively, let us resume product launch, and then increase promotion to see if there is an opportunity to restore product activity.
July 27 (Friday)
I sent Miss Gao WeChat to ask for an email, and she gave us a 163 email address.
Subsequently, we sent the document "Loss Estimate of Frontier CNC Technology Platform Loss" to Ms. Gao via email and WeChat, and Ms. Gao promised to reply to the progress on the same day.
In the evening, Miss Gao told us that the email had been forwarded to the relevant departments of the company, because the matter was special, they needed some time to evaluate, and we needed to wait patiently.
July 28 (Sat)
At 12:20, Miss Gao called and asked if there was any basis or fact that they needed to use for assessment of the losses assessed in the document "Loss Estimate of Frontier CNC Technology Platform". Our opinion is that we will provide relevant evidence and facts when necessary, and the key now is what Tencent Cloud's compensation opinion is and whether we agree with our plan.
Miss Gao asked us to wait patiently, and she would report her opinions.
July 29 (Sun)
At 12:15, Miss Gao called and said that Tencent Cloud's general manager of major customer after-sales and the backbone of the marketing department would arrange to visit our company next Monday afternoon. Miss Gao set up a WeChat group.
We instantly felt the sincerity of Tencent Cloud at that time, and felt that it might be more reliable this time, so we arranged to meet and communicate.
July 30 (Mon)
Mr. Xu, general manager of Tencent Cloud's key customer after-sales, and two Tencent Cloud entourage personnel came to our company together. Mr. Xu expressed his apologies, explaining that the purpose of their visit was to help us quickly restore the products that had been discontinued.
All the data is lost, how can it be recovered quickly?
After understanding the situation, they realized that the consequences of this data were more serious, and the product could not resume operation quickly, and they needed time to go back and discuss it, and promised to give a specific compensation plan on Thursday.
July 31 - August 1 (Tuesday - Wednesday)
Wait for Tencent Cloud to give the final compensation plan.
Thursday, August 2
We urged Mr. Xu in the WeChat group to give a compensation plan today, and the reply was sent to us in the afternoon and communicated by phone.
At 18:17 p.m., I received a call from Miss Tencent Cloud Gao, explaining that their compensation plan is: the total compensation liability does not exceed the total service fee charged by Tencent Cloud for the default service, and a package worth 100,000 yuan of Tencent Cloud is given.
Hearing such an answer, we almost collapsed instantly! Subsequently, Tencent Cloud seemed to feel that there was something wrong with such a reply, and quickly changed the 100,000 package package in the compensation plan to 132,900 yuan in cash.
At the threshold of the company's life and death, Tencent Cloud keeps saying that it attaches importance to it, and they will be responsible for the accident, and we also hope that Tencent Cloud can provide reasonable compensation funds to return the first line of life of the startup. After waiting for more than ten days, the result was a pitiful amount of compensation.....
We have also strengthened our determination to defend our rights, we believe that there is a sense of justice in this society, and the law can protect property from loss, even if an egg touches a stone, it must be touched.... This also involves many professional issues, is it an exaggeration of the fact that Tencent Cloud's claimed 99.9999999% data reliability is equipped with a three-copy storage strategy for EVS disks? Is Tencent Cloud's so-called cloud server a "cloud" in the true sense? Is Tencent Cloud's so-called 3 backups real? These are worth thinking and discussing in the industry...
|
Posted on 8/6/2018 4:30:08 PM
|
|
|
|
