Mongo db world 2014 billrun

MongoDB World 2014
by S.D.O.C. Ltd.
Billing on Top of MongoDB
Ofer Cohen

Who Am I ?
● Open source evangelist
● Former Board member of OpenSourceMatters
(Non-profit-org behind Joomla)
● S.D.O.C. Ltd. Co-Founder

What are we doing?
● We increase business success through great
open source technologies and services
○ Open & Transparent
○ Don't reinvent the wheel
○ High Quality
○ Lean & Effective

How did we get to billing (history)
● Client: Golan Telecom (Israel)

● Client: Golan Telecom
○ New & lean player in the market
○ 0=>~1M subscribers in 4 years
○ Limited resources & love open source

● Client: Golan Telecom
○ Start environment from Day 1
○ Short and aggressive time to market
○ Unlimited plan for ~25$
○ Customer can do almost everything in the website
■ Obviously requires 24/7 uptime

● Start with Anti-Fraud solution
● 2 Different data structure, 2 separated tables
○ Outgoing calls (MOC)
○ incoming calls (MTC)
○ SMS (duration=0 means SMS)

Anti-Fraud in RDBMS
How was it look like? Example code...
$base_query = "SELECT imsi FROM moc WHERE callEventStartTimeStamp >=" . $start
. " UNION SELECT imsi FROM mtc WHERE callEventStartTimeStamp >=" . $start
$base_query = "SELECT imsi FROM (" . $base_query . ") AS qry ";
if (isset($args['imsi']))
$base_query .= "WHERE imsi = '" . $this->_connection->real_escape_string($args['imsi']) . "'";
$base_query .= "GROUP BY imsi ";
$mtc_join_query = "SELECT 'mtc' AS type, imsi, SUM(callEventDuration) AS duration, SUM(CEILING(callEventDuration/60)*60) AS duration_round "
. ", SUM(chargeAmount) charge "
. ", SUM(IF(SUBSTRING(callingNumber, 1, 3)='972', callEventDuration, IF(CHAR_LENGTH(callingNumber)<=10, callEventDuration, 0))) AS israel_duration "
. ", SUM(IF(SUBSTRING(callingNumber, 1, 3)='972', CEILING(callEventDuration/60)*60, IF(CHAR_LENGTH(callingNumber)<=10, CEILING(callEventDuration/60)*60, 0))) AS
israel_duration_round "
. ", SUM(IF(SUBSTRING(callingNumber, 1, 3)!='972', IF(CHAR_LENGTH(callingNumber)>10, callEventDuration, 0), 0)) AS non_israel_duration "
. ", SUM(IF(SUBSTRING(callingNumber, 1, 3)!='972', IF(CHAR_LENGTH(callingNumber)>10, CEILING(callEventDuration/60)*60, 0), 0)) AS non_israel_duration_round "
. ", SUM(IF(callEventDuration = 0, 1, 0)) AS sms_count "
. "FROM mtc "
. "WHERE callEventStartTimeStamp >=" . $start . " "
. "GROUP BY type, imsi";

Anti-Fraud in RDBMS
$moc_join_query = "SELECT 'moc' AS type, imsi, SUM(callEventDuration) AS duration, SUM(CEILING(callEventDuration/60)*60) AS duration_round "
. ", SUM(chargeAmount) charge "
. ", SUM(IF(SUBSTRING(connectedNumber, 1, 3)='972', callEventDuration, IF(CHAR_LENGTH(connectedNumber)<=10, callEventDuration, 0))) AS israel_duration "
. ", SUM(IF(SUBSTRING(connectedNumber, 1, 3)='972', CEILING(callEventDuration/60)*60, IF(CHAR_LENGTH(connectedNumber)<=10, CEILING(callEventDuration/60)*60, 0))) AS israel_duration_round "
. ", SUM(IF(SUBSTRING(connectedNumber, 1, 3)!='972', IF(CHAR_LENGTH(connectedNumber)>10, callEventDuration, 0), 0)) AS non_israel_duration "
. ", SUM(IF(SUBSTRING(connectedNumber, 1, 3)!='972', IF(CHAR_LENGTH(connectedNumber)>10, CEILING(callEventDuration/60)*60, 0), 0)) AS non_israel_duration_round "
. ", SUM(IF(callEventDuration = 0, 1, 0)) AS sms_count "
. "FROM moc "
. "WHERE callEventStartTimeStamp >=" . $start . " "
. "GROUP BY type, imsi";

Anti-Fraud in RDBMS
$group_query = "SELECT base.imsi, moc.duration AS moc_duration, moc.charge AS moc_charge, "
. "mtc.duration AS mtc_duration, mtc.charge AS mtc_charge, "
. "mtc.duration_round AS mtc_duration_round, moc.duration_round AS moc_duration_round, "
. "moc.israel_duration AS moc_israel_duration, moc.non_israel_duration AS moc_non_israel_duration, "
. "moc.israel_duration_round AS moc_israel_duration_round, moc.non_israel_duration_round AS moc_non_israel_duration_round, "
. "mtc.israel_duration AS mtc_israel_duration, mtc.non_israel_duration AS mtc_non_israel_duration, "
. "mtc.israel_duration_round AS mtc_israel_duration_round, mtc.non_israel_duration_round AS mtc_non_israel_duration_round, "
. "mtc.sms_count AS mtc_sms_count, moc.sms_count AS moc_sms_count "
. "FROM "
. "( " . $base_query . " ) AS base "
. " LEFT JOIN (" . $mtc_join_query . " ) AS mtc ON base.imsi = mtc.imsi "
. " LEFT JOIN (" . $moc_join_query . " ) AS moc ON base.imsi = moc.imsi " ;
if (isset($args['limit'])) {
$limit = (int) $args['limit'];
} else {
$limit = 100000;
}

Anti-Fraud in RDBMS
How was it feel…?

After moving to Mongo
● One main collection for 2 types
● Aggregate much more simple

$base_match = array(
'$match' => array(
'source' => 'nrtrde',
'unified_record_time' => array('$gte' => new MongoDate($charge_time)),
)
);
$where = array(
'$match' => array(
'record_type' => 'MOC',
'connectedNumber' => array('$regex' => '^972'),
'event_stamp' => array('$exists' => false),
'deposit_stamp' => array('$exists' => false),
'callEventDurationRound' => array('$gt' => 0), // not sms
),
);
$group = array(
'$group' => array(
"_id" => '$imsi',
"moc_israel" => array('$sum' => '$callEventDurationRound'),
'lines_stamps' => array('$addToSet' => '$stamp'),
),
);
$project = array(
'$project' => array(
'imsi' => '$_id',
'_id' => 0,
'moc_israel' => 1,
'lines_stamps' => 1,
),
);

$having = array(
'$match' => array(
'moc_israel' => array('$gte' => Billrun_Factory::config()->getConfigValue('nrtrde.thresholds.moc.israel'))
),
);
$moc_israel = $lines->aggregate($base_match, $where, $group, $project, $having);
//sms out to all numbers
$where['$match']['record_type'] = 'MOC';
$where['$match']['callEventDurationRound'] = 0;
$group['$group']['sms_out'] = $group['$group']['mtc_all'];
unset($group['$group']['mtc_all']);
unset($having['$match']['mtc_all']);
$group['$group']['sms_out'] = array('$sum' => 1);
$having['$match']['sms_out'] = array('$gte' => Billrun_Factory::config()->getConfigValue('nrtrde.thresholds.smsout'));
$project['$project']['sms_out'] = 1;
unset($project['$project']['mtc_all']);
$sms_out = $lines->aggregate($base_match, $where, $group, $project, $having);

What is billing?
● Group of processes of communications
service providers
● responsible to collect consumption data
● calculate charging and billing information
● produce bills to customers
● process their payments and manage debt
collection
Wikipedia

What is billing?
● This is how we see it
● The KISS way

Telecom Today - RDBMS challenges
● Telecom situation world wide today:
○ Unlimited packages, extend 3g usage, with high
competition
○ High-volume - 4g, LTE
○ Different Events - different data structure

5 *main* data-structures from different sources
● NSN - Calls
● SMSC - SMS
● MMSC - MMS
● GGSN - Data
● TAP3 - International usage
Billing and MongoDB - Pros

NSN Record
> db.lines.findOne({"usaget" : "call", aid:XXXXXXX})
{
"_id" : ObjectId("52bafd818f7ac3943a8b96dc"),
"stamp" : "87cea5dec484c8f6a19e44e77a2e15b4",
"record_type" : "01",
"record_number" : "13857000",
"record_status" : 0,
"exchange_id" : "000006270000",
"call_reference" : "700227d9a3",
"urt" : ISODate("2013-12-25T15:09:15Z"),
"charging_start_time" : "20131225170915",
"charging_end_time" : "20131225171517",
"call_reference_time" : "20131225170914",
"duration" : 362,
"calling_number" : "972546918666",
"called_number" : "26366667",
"call_type" : "3",
"chrg_type" : "0",
"called_imsi" : "000030000000000",
"imsi" : "000089000101076",
"in_circuit_group_name" : "",
"in_circuit_group" : "",
"out_circuit_group_name" : "NBZQZA8",
"out_circuit_group" : "0503",
"tariff_class" : "000000",
"called_number_ton" : "6",
"org_dur" : 362,
"type" : "nsn",
"source" : "binary",
"file" : "CF0322.DAT",
"log_stamp" : "0a3ffdb7c9ccc175e38be2fcc00f8c28",
"process_time" : "2013-12-25 17:35:22",
"usaget" : "call",
"usagev" : 362,
"arate" : DBRef("rates", ObjectId("521e07fcd88db0e73f0001c9")),
"aid" : XXXXXXX,
"sid" : YYYYY,
"plan" : "LARGE",
"aprice" : 0,
}

SMS Record
> db.lines.findOne({"usaget" : "sms", aid:XXXXXXX})
{
"_id" : ObjectId("52e52ff1d88db071648b4ad7"),
"stamp" : "9328f3aaa114aaba910910053a11b3e8",
"record_type" : "1",
"calling_number" : "000972546918666",
"calling_imsi" : "000089200000000",
"calling_msc" : "000972000000000",
"billable" : "000000000000000",
"called_number" : "000972547655380",
"called_imsi" : "425089109386379",
"called_msc" : "000972586279101",
"message_submition_time" : "140125192517",
"time_offest" : "02",
"message_delivery_time" : "140125192519",
"time_offest1" : "02",
"cause_of_terminition" : "100",
"call_reference" : "5137864939035049",
"message_length" : "050",
"concatenated" : "1",
"concatenated_from" : "09",
"source" : "separator_field_lines",
"type" : "smsc",
"log_stamp" : "a5950686e364d1400c13dd1857c3340e",
"file" : "140125192403_5735golan.cdr",
"process_time" : "2014-01-26 17:53:21",
"urt" : ISODate("2014-01-25T17:25:17Z"),
"usaget" : "sms",
"usagev" : 1,
"arate" : DBRef("rates", ObjectId("521e07fcd88db0e73f0001db")),
"aid" : XXXXXXX,
"sid" : YYYYY,
"plan" : "LARGE",
"aprice" : 0,
"usagesb" : 4,
"billrun" : "201402"
}

Data Record
> db.lines.findOne({"usaget" : "data", aid:XXXXXXX})
{
"_id" : ObjectId("539076678f7ac34a1d8b9367"),
"stamp" : "8a9f891ec85c5294c974a34653356055",
"imsi" : "400009209100000",
"served_imsi" : "400009209100000",
"ggsn_address" : "XX.XX.144.18",
"charging_id" : "2814645234",
"sgsn_address" : "XX.XX.145.9",
"served_pdp_address" : "XX.XX.237.95",
"urt" : ISODate("2014-06-05T09:34:47Z"),
"record_opening_time" : "20140605123447",
"ms_timezone" : "+03:00",
"node_id" : "GLTNGPT",
"served_msisdn" : "00002546918666",
"fbc_uplink_volume" : 61298,
"fbc_downlink_volume" : 217304,
"rating_group" : 0,
"type" : "ggsn",
"source" : "binary",
"file" : "GLTNGPT_-_0000056580.20140605_-_1251+0300",
"log_stamp" : "45a227ced1098bc76a44774eae04eb67",
"process_time" : "2014-06-05 16:39:40",
"usaget" : "data",
"usagev" : 278602,
"arate" : DBRef("rates", ObjectId("521e07fcd88db0e73f000200")),
"apr" : 0.020264916503503202,
"aid" : XXXXXXX,
"sid" : YYYYY,
"plan" : "LARGE",
"aprice" : 0,
"usagesb" : 478682116,
"billrun" : "201406"
}

TAP3 Record (intl roaming)
> db.lines.findOne({type:"tap3", aid:9073496})
{
"_id" : ObjectId("538d9ac98f7ac3e17d8b4fd6"),
"stamp" : "8f6cdc8662307ee2ed951ce640a585b5",
"basicCallInformation" : {
"GprsChargeableSubscriber" : {
"chargeableSubscriber" : {
"simChargeableSubscriber" : {
"imsi" : "400009209100000"
}
},
"pdpAddress" : "XX.XX.227.158"
},
"GprsDestination" : {
"AccessPointNameNI" : "internet.golantelecom.net.il"
},
"CallEventStartTimeStamp" : {
"localTimeStamp" : "20140529205131",
"TimeOffsetCode" : 0
},
"TotalCallEventDuration" : 163
},
"LocationInformation" : {
"gprsNetworkLocation" : {
"RecEntityCodeList" : {
"RecEntityCode" : [
"",
"u0000"
]
},
"LocationArea" : 00001,
"CellId" : 0001
},
"GeographicalLocation" : {
"ServingNetwork" : "MMMM"
}
},
"ImeiOrEsn" : false,
"GprsServiceUsed" : {
"DataVolumeIncoming" : 195120,
"DataVolumeOutgoing" : 48600,
"ChargeInformationList" : {
"ChargeInformation" : {
"ChargedItem" : "X",
"ExchangeRateCode" : 0,
"ChargeDetailList" : {
"ChargeDetail" : {
"ChargeType" : "00",
"Charge" : 001,
"ChargeableUnits" : 100000,
"ChargedUnits" : 100000,
"ChargeDetailTimeStamp" : {
"localTimeStamp" : "20140529205131",
"TimeOffset" : 0
}
}
}
}
}
},
"OperatorSpecInfoList" : {
"OperatorSpecInformation" : [
"00000000.0000",
"00000000.0000",
"00000000.0000"
]
},
"record_type" : "e",
"urt" : ISODate("2014-05-29T18:51:31Z"),
"tzoffset" : "+0200",
"imsi" : "400009209100000",
"serving_network" : "DEUE2",
"sdr" : 0.0001,
"exchange_rate" : 1.12078,
"type" : "tap3",
"file" : "CDBELHBISRGT02253",
"log_stamp" : "a0ad109c6e795f6c1feeef9ef649d937",
"process_time" : "2014-06-03 12:50:08",
"usaget" : "data",
"usagev" : 243720,
"arate" : DBRef("rates", ObjectId
("521e07fed88db0e73f000219")),
"apr" : 0.46640616,
"aid" : 9073496,
"sid" : 78288,
"plan" : "LARGE",
"out_plan" : 243720,
"aprice" : 0.46640616,
"usagesb" : 39139746,
"billrun" : "201406"
}

Loose coupling and loose traditional billing components
Old
w/o MongoDb New
with MongoDb

● Call can be separated CDRs
● BillRun unify records only on export or
presentation layer you are aggregate not in
the DB.
● No need for mediation
● Can monitor CDR level and use billing in one
system

Sophisticated rating module
● Rating module depends on CDR structure
● Easy to implement recurring rating

MongoDB advantages with Billing
Invoice JSON2PDF process
● Use json as metadata for the PDF
● Easy to export
● Fast processing

Invoice metadata
> db.billrun.findOne({billrun_key:"201405", aid:9073496})
{
"aid" : NumberLong(9073496),
"subs" : [
{
"sid" : NumberLong(78288),
"subscriber_status" : "open",
"current_plan" : DBRef("plans", ObjectId("51bd8dc9eb2f76d2178dd3dd")),
"next_plan" : DBRef("plans", ObjectId("51bd8dc9eb2f76d2178dd3dd")),
"kosher" : false,
"breakdown" : {
"in_plan" : {
"base" : {
"INTERNET_BILL_BY_VOLUME" : {
"totals" : {
"data" : {
"usagev" : NumberLong(1975547725),
"cost" : NumberLong(0),
"count" : NumberLong(1352)
}
},
"vat" : 0.18
},
"IL_MOBILE" : {
"totals" : {
"sms" : {
},
"call" : {
}
},
"vat" : 0.18
},
"IL_FIX" : {
"totals" : {
"call" : {
}
},
"vat" : 0.18
},
"INTERNAL_VOICE_MAIL_CALL" : {
"totals" : {
"call" : {
}
},
"vat" : 0.18
},
"service" : {
"cost" : 83.898305085,
"vat" : 0.18
}
},
"intl" : {
"KT_USA_NEW" : {
"totals" : {
"call" : {
}
},
"vat" : 0.18
}
}
},

Invoice metadata
"credit" : {
"refund_vatable" : {
"CRM-REFUND_PROMOTION_1024-BILLRUN_201405" : -33.898305084746
}
}
},
"lines" : {
"data" : {
"counters" : {
"20140425" : {
"aprice" : NumberLong(0),
"plan_flag" : "in"
},
…. /* data by date really long, so let’s cut it from this demonstration */
}
}
},
"totals" : {
"vatable" : 50.000000000254005,
"before_vat" : 50.000000000254005,
"after_vat" : 59.00000000029973
},
"costs" : {
"credit" : {
"refund" : {
"vatable" : -33.898305084746
}
},
"flat" : {
"vatable" : 83.898305085
}
}
},
{
"sid" : NumberLong(354961),
"subscriber_status" : "open",
"current_plan" : DBRef("plans", ObjectId("51bd8dc9eb2f76d2178dd3de")),
"next_plan" : DBRef("plans", ObjectId("51bd8dc9eb2f76d2178dd3de")),
"kosher" : false,
"breakdown" : {
"in_plan" : {
"base" : {
"service" : {
"cost" : 8.466101695,
"vat" : 0.18
}
}
}
},
"totals" : {
"vatable" : 8.466101695,
"before_vat" : 8.466101695,
"after_vat" : 9.9900000001
},
"costs" : {
"flat" : {
"vatable" : 8.466101695
}
}
}
],
"vat" : 0.18,
"billrun_key" : "201405",
"totals" : {
"before_vat" : 58.466101695254004,
"after_vat" : 68.99000000039973,
"vatable" : 58.466101695254004
},
"_id" : ObjectId("5382fd3cd88db0c31a8b74cc"),
"invoice_id" : NumberLong(14738102),
"invoice_file" : "201405_009073496_00014738102.xml"
}

Infrastructure
BETA
First Production
Today
Data center 1
Data center 2
Data center 1
Data center 2 Data center 1
Data center 2
BillRun application
BillRun core
PHP Yaf
framework
Mongodloid,
Zend and more
libraries
MongoDB
App stack
Web service Cli/Cron services

Pay Attention! What to keep in mind
Transactional (tx) processes
● write concern - acknowledged (default in 2.4)
● findAndModify (A.K.A FAM) - document tx
● value=oldValue
● 2 phase commit - app side
● Transaction lock by design

High performance tricks
● With SSD you get x20 more performance
● MongoDB loves RAM
● Follow MongoDB production notes
○ Readahead low as possible
○ THP - transparent hugepages
Pay Attention! What to keep in mind

TCO - MongoDB based solution
● 3 Months dev
● 3 Months QA
● Few days of maintenance infra and app
● Easy to scale
● Easy to add features

Thank you, Ofer Cohen S.
D.O.C. Ltd.
ofer@billrun.net @oc666
BillRun
Billing on Top of

Mongo db world 2014 billrun

More Related Content

What's hot (20)

Similar to Mongo db world 2014 billrun (20)

More from MongoDB (20)

Recently uploaded (20)

Mongo db world 2014 billrun