BSNL Dataone - Metering system b0rked

There are so many issues with BSNL's broadband usage metering system, which can be accessed at http://10.240.43.216/, which I've been trying to get resolved since I started using the service in Feb 2005, and inspite of my having brought these issues to their notice multiple times and sought clarificiations. None of their explainations are satisfactory or logical. And most of these issues have a direct impact on the billing system too.

I'm putting up this page with the hope that this would bring some clarity to the other Dataone subscribers about the state of the system and maybe with more people following up the issue, the issues might be resolved much faster!

I'll list the issues in chronological order so that it'll be easier for me to add new problems right at the bottom of this document

1. Usage monitoring site rejects all browsers except MSIE

Status: Issue still hasn't been resolved as of May 2006.

Update: on Sat Jun 17 11:23:52 IST 2006 There is a "non-ie compatible" version of the portal available. (Details here)

Sometime around April 2005, I brought it to their notice that the usage metering system works only on Internet Explorer. They were unaware of the issue and promised to look into it. And said it should be fixed soon.

The last I heard from them about this was in April 2006, when I was informed that the sytem worked on Mozilla and Netscape 8.1 browsers, but when I tested it out, it wouldn't work on any of the browsers, Mozilla/Firefox/Opera/Safari and later realised that Netscape 8.1 comes with an option for using MSIE for rendering.(Which is the only reason the site worked on that browser, and that again is a Windows only browser)

They haven't done any changes to the browser compatibility checking at all. http://10.240.43.216/check.js still says...

function isValidBrowser() {
  var s = navigator.userAgent;
  if (s.indexOf("MSIE") == -1)
  {
    return false;
  }
  var ss = s.split(" ");
  var i=0;
  var version = 0;
  for(i=0;i<ss.length;i++)
  {
    if ( ss[i]=="MSIE" ) {
      version = ss[i+1].substring(0,1);
      break;
    }
  }
  if (version < 5) {
    return false;
  }
  return true;
}

2. User sessions on Dataone cannot last for more than 24 hours.(and are uncleanly terminated.)

Status: They aren't doing anything about it.

When I finally figured out a work around to view the usage monitoring page under opera, and took a look at my usage, I found an extraoridinary number of sessions that have ended at 86400 seconds.

My initial queries with CustomerCare etc flatly denied that they'd were disconnecting non-idle users.

Upon following up the issue with some senior officials, I've been told that it's a limitation of the metering system and they cannot have sessions last longer than 24hours.

It can't really be that difficult for them to keep account of sessions longer than 24 hours, because their accounting/metering seems to be done in smaller intervals (a few minutes) and incrementally added. After BSNL isn't the only ISP in the world that bills on usage.

I did have a single session which spanned 3 days sometime in late Sept 2005, so the system is definitely capable of doing so.

While I do very much hate the 24 hour disconnects which tend to happen mostly when I'm least expecting it or too engrossed to notice that all my active sessions have hung. My main crib is about how they are going about with the disconnects. Unlike the disconnects which happen when you are idle, where the ppp session is terminated, at the end of 24 hours they disconnect the encapsulation layer which prevents pppd from logging stats for that session.

May  7 16:37:05 utopia pppoe[15961]: Session 2299 terminated -- received PADT from peer
May  7 16:37:05 utopia pppoe[15961]: Sent PADT

Ofcourse sending out PADT packets every 24 hours, doesn't prevent BSNL from taking out quarter-page ads in major newspapers claiming "NO MORE DISCONNECTIONS".

I used to regularly have pppoe sessions lasting over 9000minutes or almost a week, while I was on DIAS, so I'm actually facing more disconnections per week than I was used to before

3. Records shown on the website are not used for billing.

Status: Acknowledged by BSNL, and easily verifiable, proof below in #5

July 2005 was the first month that Dataone subscribers were billed for excess usage, and I was billed for a whopping 1831MB of excess usage, even though the usage monitoring site said that I'd only used 10107101Kbytes(9.638GB which is within my 10GB limit.)

BSNL was pretty prompt in acknowledging that the billing was in error, and since the problem affected all Dataone subscribers nationwide, there would have to be a uniform policy decision taken nationwide. And the bill wasn't corrected until much later(I think sometime in December 2005).

I was told that the reason for the discrepancy was that the data shown on the usage monitoring site isn't what is used for billing, and that is what caused the problem as sessions from June 2006 which ended on July 1st, were billing completely under July and not in part as they should have.

4. Usage records not independently verifiable.

Status: This won't change until they fix the 24-hr disconnects and give us access to the real logs that they use for billing.

I was on DIAS for over a year before moving to Dataone, and all I had to do to verify usage was pipe my logs through grep/cut/bc and I had the totals which were very close to what was shown on the DIAS accounting page. The variance was usually in in range of of 1-2MB(higher at my end). which was pretty acceptable since some of the services on my linux box tend to generate traffic on the public interface which never leaves my machine.

But with dataone there is no independent way to verify the usage if your sessions last 24 hours.

Customers have no choice but to rely on BSNL's metering system, but the system itself isn't anywhere close to being accurate.

Same goes for the free usage too considering the reliability(or lack thereof) of the system, it's safest to disconnect and reconnect at 2AM and 8AM to avoid billing errors

5. Usage as shown to the customer has *NEVER* matched the actual billed usage.

Status: No satisfactory explaination, except for July(which may not be right either)

In addition to what is stated in #3 above, the actually billed usage has *NEVER* matched what is shown on the usage monitoring site.

All numbers below are from BSNL, BLUE, from the usage monitoring site and RED from BSNL Account Dept., these numbers are available for the asking with their accounts department but are not printed on the bills. I got them from the CAO at my local BSNL Office

Month           Send    Recv            Gross

July 2005                               10107101KB
                                        9870
Billed          1567    10504           12071

Aug 2005                                13802392KB
                                        13478
Billed   1-15    183     882
        16-31   2188    8700            11953

Sep 2005        1905845 13201787        15107632KB
                1861    12892           14753
Billed          1859    12890           14749

Oct 2005          I didn't save my usage records for

Nov 2005          October and November unfortunately

Dec 2005        2564423 9794324         12358747KB
                2504    9564            12069
Billed          2323    8577            10900

Jan 2005        2275492 11915011        14190503KB
                2222    11635           13857
Billed          2172    11151           13323

Feb 2005        2226521 10270500        12497021KB
               2174     10029           12204
Billed         2132     9979            12111
Mar 2005        2215357 11503327        13718684KB
               2163     11233           13397
Billed         2241     11574           13815

Here is a slightly censored version of the printout I got from them.

Note: This is just for the gross usage, their free usage calculation system is probably a whole other can of worms, I've not had the time to correlate data I have on my usage pattern with BSNLs, But I have seen records which are way off the mark

6. No access to the real records used for billing

Status: A pretty firm NO so far, if I can find out who has the authority to release those logs then it is a possibility.

When I came to know that there were two sets of records(see #2 above), and they didn't necessarily match, I promptly asked for a copy of the exact records used for billing. But they just ignored my request for many months then they said they'd forwarded it to Pune and hadn't heard back. After 3 billing disputes all of which were settled in my favour, Sometime in January, which is when I first got the billing details show in #5 above. I started following up with them for the actual usage details and also details of the free usage in terms of interface stats/markers at 2AM / 8AM respectively.

I've tried my best to get these details, even cut down on my demand from "all available records" to "one month" and even down to a specific period of about 1 week in January. And even agreed to sign an NDA/not take the data out of their office etc. But to no avail.

Finally last month I was told by one official that though he had the details he wasn't authorised to release it to me, The DGM-NS who seems to be the seniormost person in-charge of all things Dataone in Bangalore has outright refused to let me have access to *my* usage/billing records.

So the current situation is, that for nearly a year now they have been billing customers without ever justifying or providing the actual usage details to them.

7. Compounding points 5 & 6 above....

Status: Fairly logical, make your own decisions.

Every single BSNL customer has probably paid for bandwidth he hasn't used...

Even if you have never been billed for excess usage, you have probably been misled by their usage monitoring system into cutting back on your usage ended up paying for bandwidth that you could very well have used.

If you have ever been billed for excess usage, you have most probably paid more than you should have, because every single time BSNL has said I've been over the prescribed limit for my plan, they have been wrong. That's 4 times in 9 bills.(The latest being the bill dated 07/04/2006, where I was billed for 294MB of excess usage which I hadn't used.)

8. Number of actual ppp sessions doesn't match number of sessions shown!

Status: I thought they fixed it in Feb.06 but it's back again.

Wouldn't it be nice if you could just look at your PPP logs and the usage monitoring site and verify if the details shown on the site atleast match those few sessions that were recorded by your machine?

Ofcourse it would be, but where is the fun in something as mundane as that ? So enter BSNL with their ability to split each REAL ppp session into multiple virtual sessions at random intervals, and possibly not for all sessions either.

Month		Real		BSNLs
		Sessions	Counter
December 05	55		142
January 06	52		110
February 06	45		47
March 06	54		67
April 06	77		128

Note:February probably has a 1-1 mapping, the extra 2 sessions are possibly the fake ones at the beginning/end of the month.

If you are looking for rhyme/reason, try the wild outdoors!

9. Usage details from website don't match ppp logs

Status: May or may not be a real issue.

Hey! I'm really surprised you are still reading this ... I'm almost tired of writing this stuff!

I'm not sure if this is a real problem or if my numbers match with those used for billing since they won't let me see the billing numbers. But they definitely don't match what is shown on their usage monitoring site.

So inspite of all the fun challenges and obstacles put forward by BSNL if you go ahead a collate your records with what you got from BSNL's usage monitoring site. This is what you'll get.

Collated Logs from BSNL and my computer - HTML.gz - MS Excel - OOo Calc

The coloured records are from my computer, and represent real ppp/pppoe sessions which lasted less than 24 hours, they are few and really spaced out thanks mostly to BSNL's 24hr disconnects. The session times tend to be off by a few seconds because pppd stores time in "decimal minutes" and i had to convert into seconds.

That table/spreadsheet is so complex I've not even bothered to try and link virtual sessions to real ones, basically look for sessions adjacent to the colored rows and if the duration varies by a few seconds, see the variance between totals.

Here are some sample records ...

Session Ending at	BSNL		pppd
12/20/05 07:33 AM	974217		911538		
12/21/05 07:50 AM	86264		77279		
12/28/05 07:50 AM	1623917		1517500		
01/01/06 07:34 AM	562619		524901
01/12/06 07:52 AM	136611		132896
01/21/06 08:49 AM	2542200		2384666
01/23/06 05:08 PM	288073		269069
01/24/06 10:55 AM	1542877		1461766
02/28/06 12:24 PM	778776		730829

10. Case of the disappearing records.

Status: Still waiting, Too recent for a BSNL-speed response.

Since December I started saving the records to a .csv file during the course of the month to get an idea of my free usage.(Since their usage monitoring system doesn't give any totals for free/billable usage).

I just noticed that I have 128 records in my copy of the file, while their website now shows only 125!

The following 3 records have disappeared from their system in the last 15-20 days!

Plan-HOME 3300, Speed-1 Mbps    2006/04/01 01:56:46     2006/04/01 08:07:17     174054  614744  788798  22231   32763
Plan-HOME 3300, Speed-1 Mbps    2006/04/01 00:07:58     2006/04/01 01:56:42     3508    4880    8388    6524    8388
Plan-HOME 3300, Speed-1 Mbps    2006/04/01 00:00:00     2006/04/01 00:07:55     3841    16056   19897   475     19897

So the totals from all records generated on that site in April is different from the number of April records currently available, which is again going to be different from what I'll be billed in my Bill for the month, and in all probability very different from the actual amount of bandwidth I've consumed in April.

If you didn't understand that go read it again, It's all so simple, elegant and non-confusing that everyone at BSNL understands exactly how things work! Right ? :-)

Update: on Tue May 16 16:26:04 IST 2006

This record has now crept into usage details for April, which wasn't there 2006-05-08 14:30:24 and three older missing records are back into the usage details too!

Plan-HOME 3300, Speed-1 Mbps    2006/04/30 14:02:15     2006/04/30 18:32:55     2523    6362    8885   16240    8885

So according to the usage metering system, for the month of April.

08 May 2006 - 125 records
16 May 2006 - 129 records

And I was billed for the month of April on 07/05/2006!

Last updated on Tue May 16 16:26:04 IST 2006