SlideShare a Scribd company logo
Ruby  application based on http www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Who am I? www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   Richard Huang a.k.a flyerhzm Work at Ekohe http://huangzhimin.com @flyerhzm http://github.com/flyerhzm
www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Http Server www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   Oauth JSON XML Params Signature Web Service Restful Cloud Computing HTML XHTML
Http protocol www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   Request - Response Header - Body Client (Browser) RubyConfChina Server Get /session/new Return session/new page Post /session Login success, redirect to / Get / Return home page
Ruby API? Official? Community? www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Get data from third-party website www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   if  the  website  provides an api if the website provides an official/community ruby api use the ruby api else write the ruby api by your self end else write the ruby codes act as a http client (browser) end
What if fetch contacts from gmail? What if fetch timelines from twitter? What if fetch friends from facebook? What if fetch contacts from msn? What if fetch repositories from github? Lucky, there are ruby api to do theses! www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
What if fetch contacts from 163 mail? What if fetch weather from sina? What if fetch friends from kaixin001? What if fetch contacts from fetion? T here are no ruby api to do theses! www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Act as  a http client (browser) Analyze the packets sent by http client (browser) in sniffer tools. Resend same packets by Ruby. www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Sniffer www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Wireshark Detect any protocol (ssh, ftp, tcp, http, …) Cross platform (Windows, Linux, Mac OS) Both raw data and printable text Not easy to view https encrypted data www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Wireshark www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   filter http one http request TCP/IP HTTP HTTP Header/Body Raw Data and Printable text
Wireshark www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   one http response
HTTP Analyzer Only detect http/https protocol Concentrated on http protocol (request timing, cookie, …) Easy to view https encrypted data Windows version only www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
HTTP Analyzer www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   http request header http response header
HTTP Analyzer www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   http response body
HTTP Analyzer www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   http request timing
Get session/new www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   Get /session/new HTTP/1.1\r\n Host: rubyconfchina.org\r\n Connect: keep-alive\r\n User-Agent: Mozilla/5.0 AppleWebKit/553.4\r\n Accept: text/html,application/xhtml+xml,text/html;q=0.9\r\n Accept-Encoding: gzip,deflate\r\n Accept-Language: en-US,en;q=0.8\r\n Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n \r\n
Get session/new response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   HTTP/1.1 200 OK\r\n Server: nginx/0.5.32\r\n Date: Sun, 20 Jun 2010 09:25:08 GMT\r\n Content-Type: text/html; charset=utf-8\r\n Transfer-Encoding: chunked\r\n Connection: keep-alive\r\n Cache-Control: private, max-age=0, must-revalidate\r\n Content-Encoding: gzip\r\n \r\n <!DOCTYPE html PUBLIC …>\n <html>\n …
Post /session www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   Post /session HTTP/1.1\r\n Host: rubyconfchina.org\r\n Connect: keep-alive\r\n User-Agent: Mozilla/5.0 AppleWebKit/553.4\r\n Referer: http://rubyconfchina.org/session/new\r\n Content-Length: 106\r\n Content-Type: application/x-www-form-urlencoded\r\n Accept: text/html,application/xhtml+xml,text/html;q=0.9\r\n \r\n authenticity_token=…&login=flyerhzm&password=… &commit=Log+in
Post /session response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   HTTP/1.1 302 Moved Temporarily\r\n Server: nginx/0.5.32\r\n Date: Sun, 20 Jun 2010 09:25:31 GMT\r\n Content-Type: text/html; charset=utf-8\r\n Connect: keep-alive\r\n Location: http://rubyconfchina.org/\r\n Cache-Control: no-cache\r\n Set-Cookie: auth_token=; _euruko_session=BAh7CTo\r\n Content-Length: 91\r\n \r\n <html><body>You are being  <a href=http://rubyconfchina.org/>redirected</a>. </body></html>
Get / www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   Get / HTTP/1.1\r\n Host: rubyconfchina.org\r\n Connect: keep-alive\r\n User-Agent: Mozilla/5.0 AppleWebKit/553.4\r\n Accept: text/html,application/xhtml+xml,text/html;q=0.9\r\n Accept-Encoding: gzip,deflate\r\n Accept-Language: en-US,en;q=0.8\r\n Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n Cookie: auth_token=; _euruko_session=BAh7CTo\r\n \r\n
Http protocol www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   HTTP/1.1 200 OK\r\n Server: nginx/0.5.32\r\n Date: Sun, 20 Jun 2010 09:25:08 GMT\r\n Content-Type: text/html; charset=utf-8\r\n Transfer-Encoding: chunked\r\n Connection: keep-alive\r\n Cache-Control: private, max-age=0, must-revalidate\r\n Content-Encoding: gzip\r\n \r\n <!DOCTYPE html PUBLIC …>\n <html>\n …
Resend same packets by Ruby www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Resend same packets by Ruby Send a GET request www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   uri = URI.parse(&quot;http://rubyconfchina.org/session/new&quot;) http = Net::HTTP.new(uri.host, uri.port) headers = { &quot;User-Agent&quot; => &quot; Mozilla/5.0 AppleWebKit/553.4 &quot;, &quot;Accept&quot; => &quot;…&quot; } response = http.request_get(uri.request_uri, headers)  puts response.body =~ /Login/  # 1942
Resend same packets by Ruby Parse GET response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   raise Exception, &quot;get response should success&quot; unless Net::HTTPSuccess === repsonse if response.body =~ %r(<input name=&quot;authenticity_token&quot; .*?value=&quot;(.*>)&quot; />)m authenticity_token = $1 end
Resend same packets by Ruby Send a POST request www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   uri = URI.parse(&quot;http://rubyconfchina.org/session&quot;) http = Net::HTTP.new(uri.host, uri.port) body = &quot;authenticity_token=#{authenticity_token}& login=flyerhzm&password=…&quot; headers = { &quot;User-Agent&quot; => &quot; Mozilla/5.0 AppleWebKit/553.4 &quot;, &quot;Accept&quot; => &quot;…&quot; } response = http.request_post(uri.request_uri, body, headers)
Resend same packets by Ruby Parse POST response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   raise Exception, &quot;get response should be redirect&quot; unless Net::HTTPRedirection === repsonse cookie = response[&quot;Set-Cookie&quot;] cookie = parse_cookie(cookie) Location = response[&quot;Location&quot;]
Resend same packets by Ruby Send a GET request www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   uri = URI.parse(Location) http = Net::HTTP.new(uri.host, uri.port) headers = { &quot;User-Agent&quot; => &quot; Mozilla/5.0 AppleWebKit/553.4 &quot;, &quot;Accept&quot; => &quot;…&quot;, &quot;Cookie&quot; => cookie } response = http.request_get(uri.request_uri, headers)  raise Exception, &quot;get repsonse should success&quot; unless Net::HTTPSuccess === response puts response.body =~ /Login/  # nil
Facility www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Mechanize A ruby library that makes automated web interaction easy.  Automatically location redirection Automatically cookies management www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Mechanize www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   a = Mechanize.new do |agent| agent.user_agent_alias = 'Linux Firefox' end a.get('http://rubyconfchina.org/session/new') do |page| puts page.body =~ /Login/  # 1942 home_page = page.form_with(:action => '/session') do |f| f.login = 'flyerhzm' f.password = '…' end.click_button puts home_page.body =~ /Login/  # nil end
Test www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Test Test http requests Test http responses www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Test request www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   Client Server GET /ht/sd.aspx?i=1 HTTP/1.1\r\n Host: 221.176.31.39\r\n … \r\n \r\n R fetion.com.cn SIP-C/4.0 F: 730020377 I: 1 Q: 1 R CN: 19D28D4978125CAA4F6E54277BA7D9EF CL: type=&quot;pc&quot; ,version=&quot;3.6.2020&quot; SIPP sid = 730020377 i = 0
Test request www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   @fetion.stubs(:sid).returns(730020377) @fetion.stubs(:i).returns(0) @fetion.stubs(:guid).returns(&quot;19D28D4978125CAA4F6E54277BA7D9EF&quot;) expected_sipc_message  =<<-EOF R fetion.com.cn SIP-C/4.0 F: 730020377 I: 1 Q: 1 R CN: 19D28D4978125CAA4F6E54277BA7D9EF CL: type=&quot;pc&quot; ,version=&quot;3.6.2020&quot; SIPP EOF expected_sipc_message.gsub!(&quot;\n&quot;, &quot;\r\n&quot;).chomp! SipcMessage.register_first(@fetion).should ==  expected_sipc_message   Copied from Sniffer tool
Test response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   Client Server HTTP/1.1 200 OK\r\n Server: nginx/0.5.32\r\n … \r\n Set-Cookie:  ssic=CBIOAAAm+FiuQgpcnFi; path=/\r\n \r\n <?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; ?> <results status-code=&quot;200&quot;> <user uri=&quot;sip:730020377@fetion.com.cn;p=6907&quot; /> </results>
Fakeweb A test helper for faking responses to web requests. No need to  modify existing code No need to write extensive stubs www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Test response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   FakeWeb.register_uri( :get, 'https://uid.fetion.com.cn/ssiportal/SSIAppSignInV4.aspx? mobileno=15800681509&digest=79cd56b93f21298dc8ae9d26de1258e3d', :body => %Q|<?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; ?><results status-code=&quot;200&quot;><user uri=&quot;sip:730020377@fetion.com.cn;p=6907&quot; /></results>|,  :set_cookie => Q|ssic=CBIOAAAm+FiuQgpcnFi; path=/| ) @fetion.login @fetion.status_code.should == &quot;200&quot; @fetion.sid.should == &quot;730020377&quot; Copied from Sniffer tool
Test response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   FakeWeb.register_uri( :get,  'https://uid.fetion.com.cn/ssiportal/SSIAppSignInV4.aspx? mobileno=15800681509&digest=79cd56b93f21298dc8ae9d26de1258e3d',  :body => %Q|<?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; ?><results status-code=&quot;401&quot; desc=&quot;password error&quot; />|,  :status => ['401', 'password error'] ) lambda { @fetion.login }.should raise_exception(Fetion::PasswordError) Copied from Sniffer tool
Test Process Do some stubs Copy the http request/response as expected message from sniffer tools Execute the method to generate http request/respones Test with expected message TDD (optional) www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
Performance www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
net-http-persistent a thread-safe wrapper for Net::HTTP that performs persistent connections for you www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   user system total real Net::HTTP 8.410000 2.400000 10.810000 ( 17.333671) Net::HTTP::Persistent 8,110000 0.880000 8.990000 ( 12.190094)
typhoeus Parallel http requests based on libcurl and libcurl-multi 500ms response, call 20 times www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development   user system total real Net::HTTP 0.030000 0.010000 0.040000 ( 10.054327) typhoeus 0.020000 0.070000 0.090000 (  0.508817)
Q&A Thank you www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development

More Related Content

PDF
Developing cross platform desktop application with Ruby
PDF
Introduction to JRuby
PDF
Ruby on Rails Presentation
PDF
When To Use Ruby On Rails
ODP
Ruby on Rails
PDF
Ruby On Rails Basics
PDF
Ruby On Rails Introduction
PPSX
Webpack & EcmaScript 6 (Webelement #32)
Developing cross platform desktop application with Ruby
Introduction to JRuby
Ruby on Rails Presentation
When To Use Ruby On Rails
Ruby on Rails
Ruby On Rails Basics
Ruby On Rails Introduction
Webpack & EcmaScript 6 (Webelement #32)

What's hot (20)

PDF
Ruby on Rails for beginners
ODP
PPTX
Ruby Programming Language - Introduction
PDF
Web Development with Python and Django
PDF
On Web Browsers
PDF
A Recovering Java Developer Learns to Go
PPTX
RJB - another choice for Ruby and Java interoperability
PDF
mod_php vs FastCGI vs FPM vs CLI
KEY
Ruby on Rails survival guide of an aged Java developer
PDF
Introduction to rails
PDF
Introduction to Ruby
PDF
RubyMotion Introduction
ODP
Asynchronous I/O in NodeJS - new standard or challenges?
PPT
Ruby Basics
 
KEY
Rack Middleware
PDF
A bridge between php and ruby
KEY
MacRuby: What is it? and why should you care?
PDF
Ruby - a tester's best friend
PPTX
Jedi Mind Tricks for Git
PDF
Let's Learn Ruby - Basic
Ruby on Rails for beginners
Ruby Programming Language - Introduction
Web Development with Python and Django
On Web Browsers
A Recovering Java Developer Learns to Go
RJB - another choice for Ruby and Java interoperability
mod_php vs FastCGI vs FPM vs CLI
Ruby on Rails survival guide of an aged Java developer
Introduction to rails
Introduction to Ruby
RubyMotion Introduction
Asynchronous I/O in NodeJS - new standard or challenges?
Ruby Basics
 
Rack Middleware
A bridge between php and ruby
MacRuby: What is it? and why should you care?
Ruby - a tester's best friend
Jedi Mind Tricks for Git
Let's Learn Ruby - Basic
Ad

Viewers also liked (20)

PPTX
Rails Request & Middlewares
PPT
Rails Request Response Lifecycle
PDF
Request-Response Cycle of Ruby on Rails App
PPTX
Web : Request-Response Lifecycle (Ruby on Rails)
PPTX
Using puppet, foreman and git to develop and operate a large scale internet s...
PDF
Mapping, Interlinking and Exposing MusicBrainz as Linked Data
PDF
React.js & Om: A hands-on walkthrough of better ways to build web UIs
PDF
Continuously-Integrated Puppet in a Dynamic Environment
PDF
JSON and the APInauts
PDF
Better encryption & security with MariaDB 10.1 & MySQL 5.7
PDF
PPTX
How We Do DevOps at Walmart: OneOps OSS Application Lifecycle Management Plat...
PDF
vSphere APIs for performance monitoring
PDF
Dlsecyx pgroammr (Dyslexic Programmer - cool stuff for scaling)
KEY
Make GUI Apps with Shoes
PDF
PostgreSQL Materialized Views with Active Record
PDF
The Complete MariaDB Server Tutorial - Percona Live 2015
PDF
Redis — The AK-47 of Post-relational Databases
PDF
Taking Control of Chaos with Docker and Puppet
PDF
Deployment with Ruby on Rails
Rails Request & Middlewares
Rails Request Response Lifecycle
Request-Response Cycle of Ruby on Rails App
Web : Request-Response Lifecycle (Ruby on Rails)
Using puppet, foreman and git to develop and operate a large scale internet s...
Mapping, Interlinking and Exposing MusicBrainz as Linked Data
React.js & Om: A hands-on walkthrough of better ways to build web UIs
Continuously-Integrated Puppet in a Dynamic Environment
JSON and the APInauts
Better encryption & security with MariaDB 10.1 & MySQL 5.7
How We Do DevOps at Walmart: OneOps OSS Application Lifecycle Management Plat...
vSphere APIs for performance monitoring
Dlsecyx pgroammr (Dyslexic Programmer - cool stuff for scaling)
Make GUI Apps with Shoes
PostgreSQL Materialized Views with Active Record
The Complete MariaDB Server Tutorial - Percona Live 2015
Redis — The AK-47 of Post-relational Databases
Taking Control of Chaos with Docker and Puppet
Deployment with Ruby on Rails
Ad

Similar to Ruby application based on http (20)

PPT
Contributing To Rails By Plugin Gem
PPT
Design Pattern From Java To Ruby
KEY
Picking gem ruby for penetration testers
KEY
Using and scaling Rack and Rack-based middleware
PPT
Rails bestpractices.com
PDF
Using ArcGIS Server with Ruby on Rails
PDF
8 Minutes On Rack
PDF
faastCrystal
PDF
KEY
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
PDF
Ruby HTTP clients comparison
KEY
FxUG@北陸資料
PDF
Rack Middleware
KEY
Rails Presentation (Anton Dmitriyev)
PDF
Services web RESTful
PDF
Middleware as Code with mruby
PDF
Ruby projects of interest for DevOps
PDF
Ricardo Sanchez - Ruby projects of interest for devops
PDF
Middleware as Code with mruby
PDF
Design Pattern From Java To Ruby
Contributing To Rails By Plugin Gem
Design Pattern From Java To Ruby
Picking gem ruby for penetration testers
Using and scaling Rack and Rack-based middleware
Rails bestpractices.com
Using ArcGIS Server with Ruby on Rails
8 Minutes On Rack
faastCrystal
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
Ruby HTTP clients comparison
FxUG@北陸資料
Rack Middleware
Rails Presentation (Anton Dmitriyev)
Services web RESTful
Middleware as Code with mruby
Ruby projects of interest for DevOps
Ricardo Sanchez - Ruby projects of interest for devops
Middleware as Code with mruby
Design Pattern From Java To Ruby

More from Richard Huang (7)

KEY
Semi Automatic Code Review
KEY
Rails project code review
KEY
rails_best practices_rubykaigi
KEY
使用Rails best practices做代码审查
PPT
Css sprite best practices
PPT
Static Code Analysis For Ruby
PPT
Email Contacts For Invitation
Semi Automatic Code Review
Rails project code review
rails_best practices_rubykaigi
使用Rails best practices做代码审查
Css sprite best practices
Static Code Analysis For Ruby
Email Contacts For Invitation

Recently uploaded (20)

PDF
Encapsulation theory and applications.pdf
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PPTX
Cloud computing and distributed systems.
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
PDF
Spectral efficient network and resource selection model in 5G networks
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
Electronic commerce courselecture one. Pdf
PPTX
Spectroscopy.pptx food analysis technology
PDF
KodekX | Application Modernization Development
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Encapsulation theory and applications.pdf
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
Dropbox Q2 2025 Financial Results & Investor Presentation
Chapter 3 Spatial Domain Image Processing.pdf
Digital-Transformation-Roadmap-for-Companies.pptx
Unlocking AI with Model Context Protocol (MCP)
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
“AI and Expert System Decision Support & Business Intelligence Systems”
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
20250228 LYD VKU AI Blended-Learning.pptx
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
Cloud computing and distributed systems.
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Per capita expenditure prediction using model stacking based on satellite ima...
Spectral efficient network and resource selection model in 5G networks
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Electronic commerce courselecture one. Pdf
Spectroscopy.pptx food analysis technology
KodekX | Application Modernization Development
Build a system with the filesystem maintained by OSTree @ COSCUP 2025

Ruby application based on http

  • 1. Ruby application based on http www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 2. Who am I? www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development Richard Huang a.k.a flyerhzm Work at Ekohe http://huangzhimin.com @flyerhzm http://github.com/flyerhzm
  • 3. www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 4. Http Server www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development Oauth JSON XML Params Signature Web Service Restful Cloud Computing HTML XHTML
  • 5. Http protocol www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development Request - Response Header - Body Client (Browser) RubyConfChina Server Get /session/new Return session/new page Post /session Login success, redirect to / Get / Return home page
  • 6. Ruby API? Official? Community? www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 7. Get data from third-party website www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development if the website provides an api if the website provides an official/community ruby api use the ruby api else write the ruby api by your self end else write the ruby codes act as a http client (browser) end
  • 8. What if fetch contacts from gmail? What if fetch timelines from twitter? What if fetch friends from facebook? What if fetch contacts from msn? What if fetch repositories from github? Lucky, there are ruby api to do theses! www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 9. What if fetch contacts from 163 mail? What if fetch weather from sina? What if fetch friends from kaixin001? What if fetch contacts from fetion? T here are no ruby api to do theses! www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 10. Act as a http client (browser) Analyze the packets sent by http client (browser) in sniffer tools. Resend same packets by Ruby. www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 11. Sniffer www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 12. Wireshark Detect any protocol (ssh, ftp, tcp, http, …) Cross platform (Windows, Linux, Mac OS) Both raw data and printable text Not easy to view https encrypted data www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 13. Wireshark www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development filter http one http request TCP/IP HTTP HTTP Header/Body Raw Data and Printable text
  • 14. Wireshark www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development one http response
  • 15. HTTP Analyzer Only detect http/https protocol Concentrated on http protocol (request timing, cookie, …) Easy to view https encrypted data Windows version only www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 16. HTTP Analyzer www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development http request header http response header
  • 17. HTTP Analyzer www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development http response body
  • 18. HTTP Analyzer www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development http request timing
  • 19. Get session/new www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development Get /session/new HTTP/1.1\r\n Host: rubyconfchina.org\r\n Connect: keep-alive\r\n User-Agent: Mozilla/5.0 AppleWebKit/553.4\r\n Accept: text/html,application/xhtml+xml,text/html;q=0.9\r\n Accept-Encoding: gzip,deflate\r\n Accept-Language: en-US,en;q=0.8\r\n Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n \r\n
  • 20. Get session/new response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development HTTP/1.1 200 OK\r\n Server: nginx/0.5.32\r\n Date: Sun, 20 Jun 2010 09:25:08 GMT\r\n Content-Type: text/html; charset=utf-8\r\n Transfer-Encoding: chunked\r\n Connection: keep-alive\r\n Cache-Control: private, max-age=0, must-revalidate\r\n Content-Encoding: gzip\r\n \r\n <!DOCTYPE html PUBLIC …>\n <html>\n …
  • 21. Post /session www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development Post /session HTTP/1.1\r\n Host: rubyconfchina.org\r\n Connect: keep-alive\r\n User-Agent: Mozilla/5.0 AppleWebKit/553.4\r\n Referer: http://rubyconfchina.org/session/new\r\n Content-Length: 106\r\n Content-Type: application/x-www-form-urlencoded\r\n Accept: text/html,application/xhtml+xml,text/html;q=0.9\r\n \r\n authenticity_token=…&login=flyerhzm&password=… &commit=Log+in
  • 22. Post /session response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development HTTP/1.1 302 Moved Temporarily\r\n Server: nginx/0.5.32\r\n Date: Sun, 20 Jun 2010 09:25:31 GMT\r\n Content-Type: text/html; charset=utf-8\r\n Connect: keep-alive\r\n Location: http://rubyconfchina.org/\r\n Cache-Control: no-cache\r\n Set-Cookie: auth_token=; _euruko_session=BAh7CTo\r\n Content-Length: 91\r\n \r\n <html><body>You are being <a href=http://rubyconfchina.org/>redirected</a>. </body></html>
  • 23. Get / www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development Get / HTTP/1.1\r\n Host: rubyconfchina.org\r\n Connect: keep-alive\r\n User-Agent: Mozilla/5.0 AppleWebKit/553.4\r\n Accept: text/html,application/xhtml+xml,text/html;q=0.9\r\n Accept-Encoding: gzip,deflate\r\n Accept-Language: en-US,en;q=0.8\r\n Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n Cookie: auth_token=; _euruko_session=BAh7CTo\r\n \r\n
  • 24. Http protocol www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development HTTP/1.1 200 OK\r\n Server: nginx/0.5.32\r\n Date: Sun, 20 Jun 2010 09:25:08 GMT\r\n Content-Type: text/html; charset=utf-8\r\n Transfer-Encoding: chunked\r\n Connection: keep-alive\r\n Cache-Control: private, max-age=0, must-revalidate\r\n Content-Encoding: gzip\r\n \r\n <!DOCTYPE html PUBLIC …>\n <html>\n …
  • 25. Resend same packets by Ruby www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 26. Resend same packets by Ruby Send a GET request www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development uri = URI.parse(&quot;http://rubyconfchina.org/session/new&quot;) http = Net::HTTP.new(uri.host, uri.port) headers = { &quot;User-Agent&quot; => &quot; Mozilla/5.0 AppleWebKit/553.4 &quot;, &quot;Accept&quot; => &quot;…&quot; } response = http.request_get(uri.request_uri, headers) puts response.body =~ /Login/ # 1942
  • 27. Resend same packets by Ruby Parse GET response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development raise Exception, &quot;get response should success&quot; unless Net::HTTPSuccess === repsonse if response.body =~ %r(<input name=&quot;authenticity_token&quot; .*?value=&quot;(.*>)&quot; />)m authenticity_token = $1 end
  • 28. Resend same packets by Ruby Send a POST request www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development uri = URI.parse(&quot;http://rubyconfchina.org/session&quot;) http = Net::HTTP.new(uri.host, uri.port) body = &quot;authenticity_token=#{authenticity_token}& login=flyerhzm&password=…&quot; headers = { &quot;User-Agent&quot; => &quot; Mozilla/5.0 AppleWebKit/553.4 &quot;, &quot;Accept&quot; => &quot;…&quot; } response = http.request_post(uri.request_uri, body, headers)
  • 29. Resend same packets by Ruby Parse POST response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development raise Exception, &quot;get response should be redirect&quot; unless Net::HTTPRedirection === repsonse cookie = response[&quot;Set-Cookie&quot;] cookie = parse_cookie(cookie) Location = response[&quot;Location&quot;]
  • 30. Resend same packets by Ruby Send a GET request www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development uri = URI.parse(Location) http = Net::HTTP.new(uri.host, uri.port) headers = { &quot;User-Agent&quot; => &quot; Mozilla/5.0 AppleWebKit/553.4 &quot;, &quot;Accept&quot; => &quot;…&quot;, &quot;Cookie&quot; => cookie } response = http.request_get(uri.request_uri, headers) raise Exception, &quot;get repsonse should success&quot; unless Net::HTTPSuccess === response puts response.body =~ /Login/ # nil
  • 31. Facility www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 32. Mechanize A ruby library that makes automated web interaction easy. Automatically location redirection Automatically cookies management www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 33. Mechanize www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development a = Mechanize.new do |agent| agent.user_agent_alias = 'Linux Firefox' end a.get('http://rubyconfchina.org/session/new') do |page| puts page.body =~ /Login/ # 1942 home_page = page.form_with(:action => '/session') do |f| f.login = 'flyerhzm' f.password = '…' end.click_button puts home_page.body =~ /Login/ # nil end
  • 34. Test www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 35. Test Test http requests Test http responses www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 36. Test request www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development Client Server GET /ht/sd.aspx?i=1 HTTP/1.1\r\n Host: 221.176.31.39\r\n … \r\n \r\n R fetion.com.cn SIP-C/4.0 F: 730020377 I: 1 Q: 1 R CN: 19D28D4978125CAA4F6E54277BA7D9EF CL: type=&quot;pc&quot; ,version=&quot;3.6.2020&quot; SIPP sid = 730020377 i = 0
  • 37. Test request www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development @fetion.stubs(:sid).returns(730020377) @fetion.stubs(:i).returns(0) @fetion.stubs(:guid).returns(&quot;19D28D4978125CAA4F6E54277BA7D9EF&quot;) expected_sipc_message =<<-EOF R fetion.com.cn SIP-C/4.0 F: 730020377 I: 1 Q: 1 R CN: 19D28D4978125CAA4F6E54277BA7D9EF CL: type=&quot;pc&quot; ,version=&quot;3.6.2020&quot; SIPP EOF expected_sipc_message.gsub!(&quot;\n&quot;, &quot;\r\n&quot;).chomp! SipcMessage.register_first(@fetion).should == expected_sipc_message Copied from Sniffer tool
  • 38. Test response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development Client Server HTTP/1.1 200 OK\r\n Server: nginx/0.5.32\r\n … \r\n Set-Cookie: ssic=CBIOAAAm+FiuQgpcnFi; path=/\r\n \r\n <?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; ?> <results status-code=&quot;200&quot;> <user uri=&quot;sip:730020377@fetion.com.cn;p=6907&quot; /> </results>
  • 39. Fakeweb A test helper for faking responses to web requests. No need to modify existing code No need to write extensive stubs www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 40. Test response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development FakeWeb.register_uri( :get, 'https://uid.fetion.com.cn/ssiportal/SSIAppSignInV4.aspx? mobileno=15800681509&digest=79cd56b93f21298dc8ae9d26de1258e3d', :body => %Q|<?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; ?><results status-code=&quot;200&quot;><user uri=&quot;sip:730020377@fetion.com.cn;p=6907&quot; /></results>|, :set_cookie => Q|ssic=CBIOAAAm+FiuQgpcnFi; path=/| ) @fetion.login @fetion.status_code.should == &quot;200&quot; @fetion.sid.should == &quot;730020377&quot; Copied from Sniffer tool
  • 41. Test response www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development FakeWeb.register_uri( :get, 'https://uid.fetion.com.cn/ssiportal/SSIAppSignInV4.aspx? mobileno=15800681509&digest=79cd56b93f21298dc8ae9d26de1258e3d', :body => %Q|<?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; ?><results status-code=&quot;401&quot; desc=&quot;password error&quot; />|, :status => ['401', 'password error'] ) lambda { @fetion.login }.should raise_exception(Fetion::PasswordError) Copied from Sniffer tool
  • 42. Test Process Do some stubs Copy the http request/response as expected message from sniffer tools Execute the method to generate http request/respones Test with expected message TDD (optional) www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 43. Performance www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development
  • 44. net-http-persistent a thread-safe wrapper for Net::HTTP that performs persistent connections for you www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development user system total real Net::HTTP 8.410000 2.400000 10.810000 ( 17.333671) Net::HTTP::Persistent 8,110000 0.880000 8.990000 ( 12.190094)
  • 45. typhoeus Parallel http requests based on libcurl and libcurl-multi 500ms response, call 20 times www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development user system total real Net::HTTP 0.030000 0.010000 0.040000 ( 10.054327) typhoeus 0.020000 0.070000 0.090000 ( 0.508817)
  • 46. Q&A Thank you www.ekohe.com Web Development & Graphic Design China Ruby on Rails Development - Rails Consulting - Rails Services - Merb - Offshore Web Development