How to discover the Ruby's defects with web application

柴田博志
SHIBATA Hiroshi
paperboy&co.
asakusa.rb
paperboy&co., Inc.
Ruby の不具合の見つけ方
発表場所 RubyConf Taiwan 2012 2012-12-8(Sat)
How to discover the
Ruby's defects with web
application.

5文字のテ
キストらし
いですよ？
大丈夫か？Xie Xie rubytaiwan. Lai tai wan, wan hen
kai xin.

SHIBATA Hiroshi(@hsbt)
Wo shi cong ri ben lai de. Wo jiao Hiroshi,
Shibata.
My name is Hiroshi SHIBATA, please call
me Hiroshi or SHIBATA. This jacket is my
trademark and ﬁghting armor. Today is
same jacket.

Thanks for @ihower and @rubytaiwan
accepting my proposal. I appreciate it. I’m
super exciting.
Taiwan is good city. I love taiwan. I
decided to arrived rubyconf taiwan in next
year.

asakusa.rb
I’m member of asakusa.rb. asakusa.rb is
inspired by Seattle.rb. This meetup start
every tuesday. this meetup founder is
Akira Matsuda. If you arrived tokyo, japan,
please join our meetup.

I’m working at paperboy&co. My job is
solving for all technical problems such as
continuous deployment, security issue and
gaining code quality etc... it’s company
wide scrum master.

http://fanic.jp
Our one of the rails service is fanic. fanic
is that you can have personal music store.
I’m sorry, fanic is only japanese language
available.

http://musiccaptains.com/zh/2012/11/27/fanic-
music-website-song-purchase/
You are lucky. Some of chinese writer
wrote introduction of fanic. this entry is
chinese!
Please join it.

tDiary
In this talk, I’ll introduce tDiary and how
tDiary can discover ruby’s defects. I don’t
talk agile and how to maintain legacy
code. If you interested these theme, please
talk to me after session.

http://github.com/tdiary
tDiary is hosted by github now, It this url.
I’m sorry this site is only available for
japanese and english information. if you
interested tDiary, please join our
organization.

ｗhat’s?
First, what’s tDiary?
Do you know tDiary? Please raise up your
hand?

Nikki
System
tDiary is Nikki System. Nikki is Diary.
Japanese internet has web nikki culture
before blog.

Like a Blog
Nikki is a blog in 2012. We don’t divide
their.

Speciﬁcation
Overview
I’ll introduce tDiary mainly speciﬁcation.

3. Pluggable
2. CGI/Rack
1. Ruby
First, it use Ruby, and this is most
important things. Second, running
environments are traditional CGI and Rack.
It means tDiary can run in all of
application server. For example apache,
passenger and unicorn. Third, it has
pluggable mechanism. It mechanism is
possibled by ruby meta-programming.
this feature supports newbie rubyists.

History
Second, I talk about history.

Ruby 1.6.7
2002
In 2002, tDiary began to development in
the sf.net and used CVS. I joined tDiary
developper team this year.
First version supported Ruby-1.6.7. Please
raise up your hands if you had used this
version of Ruby?

Ruby 1.8.0
2003
In 2003, tDiary supported Ruby-1.8.0.
This version included many standard
libraries.

What is Ruby
needed to run
the tDiary?
“What is Ruby needed to run the tDiary.”
This phrase posted some contents in
2004. It means tDiary more famous lather
than Ruby.
Everyone knows tDiary in Japan.

Ruby 1.9.0
2008
In 2008, tDiary supported 1.9.0. Big
internal changing occurred in this version.
All of String have Encoding. tDiary was
broken by encoding changes. It was so
hard for us, and we have be supporting
now.

Ruby
2.0.0
Probably, tDiary is supporting 2.0. This
feature is assigned me.

As you can see that tDiary is growing up
with Ruby.
In other hand, Ruby is growing up with
tDiary.

tDiary discovered
Ruby’s defects
In other words, tDiary discovered some of
ruby’s defects.

why?
Why tDiary be able to discover Ruby’s
defects?

Background
I’m going to describe Japanese Internet
environment. Nearly 2004, web
programmer need to use shared hosting
server for web programming.

tDiary can use plugin written by tiny ruby
code. Implementation of this feature need
to ruby-meta-programming.

eval
tDiary’s core feature is using eval,
instance_eval and class_eval.

private
# loading tdiary.conf in current directory
def configure_attrs
@secure = true unless @secure
@options = {}
eval( File::open( 'tdiary.conf' ) {|f| f.read }.untaint, b,
"(tdiary.conf)", 1 )
# language setup
@lang = 'ja' unless @lang
begin
This is part of tDiary’s code.
tDiary conﬁguration was written by Ruby
code. it’s not by yaml and json.

# The value must be the Array of Ruby and Array's contents are
String
# of Ruby. Though these Strings are converted to Regular Expression
# of Ruby when compared, you can't use Regexp of Ruby.
@no_referer = [
'^' + Regexp.quote( base_url ), # Your diary
'^http://localhost[:/]',
'^http://192.168.',
'^http://172.1[6789]',
'^http://172.2[0-9]',
'^http://172.3[01]',
'^http://10.',
]
# URLs which are only recorded into Today's Link (Regular Expression)
# @only_volatile is an array of URLs to record into only Today's
Link.
# When a referer match to this list, it will be recoreded to
volatile
# list. This list will be cleared when you make new text in new
day.
# Specify @only_volatile same style of @no_referer.
@only_volatile = [
]
# The rules which convert the specified URL to the word (Regular
Expression)
# @referer_table is configured so that readable URLs are shown in
This is example of tdiary configuration
file.
If you try to customize tDiary behavior,
you should write method definitions,
variable settings and more into
“tdiary.conf”.
It’s very useful.

private
def setup_attr_accessor_to_all_ivars
names = instance_variables().collect {|ivar| ivar.to_s.sub(/
@/, '') }
(class << self; self; end).class_eval { attr_accessor
*names }
end
def configure_bot_pattern
bot = ["bot", "spider", "antenna", "crawler", "moget",
"slurp"]
bot += @options['bot'] || []
@bot = Regexp::new( "(#{bot.uniq.join( '|' )})", true )
end
Next point is class_eval.
tDiary uses classic “cgi.rb” and creates CGI
instance per all requests.
CGI instance have a conﬁguration
instance. It needs access mechanism for
tDiary’s core and plugins.
class_eval make conﬁguration instance
possible to access it.

def load_plugin( file )
@resource_loaded = false
begin
res_file = File::dirname( file ) + "/#{@conf.lang}/"
+ File::basename( file )
open( res_file.untaint ) do |src|
instance_eval( src.read.untaint, "(plugin/
#{@conf.lang}/#{File::basename( res_file )})", 1 )
end
@resource_loaded = true
rescue IOError, Errno::ENOENT
end
File::open( file.untaint ) do |src|
instance_eval( src.read.untaint, "(plugin/
#{File::basename( file )})", 1 )
end
end
The i18n of tDiary use Ruby code. this is
same the case of conﬁguration ﬁles. tDiary
doesn’t use i18n gem! In past, tDiary
include chinese language. but we dropped
it, because we have no maintainer.

One of 2.0.0 feature is “Refinements”.
tDiary have many monkey patches for
Ruby’s core feature. I’m going to fix these
patches by using “Refinements”.

“eval” is powerful metaprogramming in
ruby. but you worry about security
issue.support two things. one, share
hosting environment. imagine of you
made cgi in shared hosting environment.
that cgi can use code written by other
people. If this code is exploit code. How
do you protect it?

$SAFE
tDiary protects with variable of $SAFE
against security issue.
I’m introducing $SAFE mechanism in Ruby.
Did you know this feature?

open( res_file.untaint ) do |
src|
instance_eval( src.read.untaint,
"(plugin/#{@conf.lang}/
#{File::basename( res_file )})",
1 )
end
@resource_loaded =
true
rescue IOError,
Errno::ENOENT
end
File::open( file.untaint ) do |
src|
instance_eval( src.read.untaint,
"(plugin/
#{File::basename( file )})", 1 )
end
end
I talk overview of $SAFE mechanism. $SAFE
mechanism makes all object taint. Once
you use $SAFE, you can’t use tainted
object in some function.
If you want to use tainted objects, you
should call “untaint” method.

$SAFE=1
$ ruby -e '$SAFE = 1; open(ARGV[0])' foo
-e:1:in `initialize': Insecure operation -
initialize (SecurityError)
from -e:1
$SAFE level 1 disallow “File.open” with
tainted strings. If you want to open with
ARGV[0] string. you should call “untaint”.

$SAFE=4
% ruby -e '$SAFE=4; eval("p :foo".untaint)'
-e:1:in `untaint': Insecure operation at level 4 (SecurityError)
from -e:1:in `<main>'
% ruby -e '$SAFE=1; eval("p :foo".untaint)'
:foo
$SAFE level 4. If you use untainted strings
with eval, this level disallow using it.
tDiary customization is handled with eval
and $SAFE mechanism.

=begin
== Safe module
=end
module Safe
def safe( level = 4 )
result = nil
if $SAFE < level then
Proc.new {
$SAFE = level
result = yield
}.call
else
result = yield
end
result
end
module_function :safe
end
It’s protips. If you use different $SAFE level
in same code. you need to use “Proc#new”
and “Proc#call”.
tDiary use this procedure when running
with shared environment.

bugreport
http://bugs.ruby-lang.org/issues/5279
fixed r33328:
* encoding.c (require_enc): reject only loading from untrusted
load paths. [ruby-dev:44541] [Bug #5279]
* transcode.c (load_transcoder_entry): ditto. assume system
default tmpdir safe. [ruby-dev:42089]
fixed r29209: assume system default tmpdir safe. [ruby-dev:42089]
* load.c (rb_require_safe): raises when the path to be loaded is
tainted. [ruby-dev:37843]
I've never seen that other people are using
$SAFE feature. therefore $SAFE feature is
very buggy.

* load.c (rb_require_safe): raises when the path to be loaded is tainted. [ruby-dev:37843]
% ruby -e '$SAFE=1;p require "zlib"'
-e:1:in `require': Insecure operation -
require (SecurityError)
% ruby -e '$SAFE=1;p require "English"'
-e:1:in `require': Insecure operation -
require (SecurityError)
This defect is that ruby raises exception
when ruby call require with tainted path.

fixed r29209: assume system default tmpdir safe. [ruby-dev:42089]
% ruby -e
'$SAFE=1;File.expand_path(".".taint)'
-e:1:in `expand_path': Insecure operation -
expand_path (SecurityError)
This defects. if you use “tmpdir” running
with $SAFE level 4, Ruby raised exception.

fixed r33328:
* encoding.c (require_enc): reject only loading from untrusted
load paths. [ruby-dev:44541] [Bug #5279]
* transcode.c (load_transcoder_entry): ditto. assume system default tmpdir safe.
[ruby-dev:42089]
% ruby -e ‘$SAFE = 3; "a".encode("UTF-16")’
-e:1:in `encode': Insecure operation - encode
(SecurityError)
In this report. If you run some code with
$SAFE level 4, and not with calling
“encode”. Ruby is raised exception.
I already talked about all of String has
Encoding in Ruby 1.9.
This defects is important, because anyone
never use String#encode with $SAFE level
4 in Ruby 1.9

5文字のテ
キストらし
いですよ？
大丈夫か？I’m sorry, I introduce bad news. JRuby isn’t
support $SAFE yet.

@shugomaeda
$SAFE users is only
tDiary. I want to drop
it in next version of
ruby.
really!?
@hsbt
I think probably dropped out $SAFE
feature in 2.1.
We accept it. so tDiary need to change
with rubies. The Internet environment is
changing now. people doesn’t use shared
hosting server. they use PaaS like a sqale
and heroku.

tDiary discover ruby’s defects is easy. In
this case is only tDiary being?
I think no.

tDiary uses Travis-CI.
I build ruby-trunk every morning, and run
it with tDiary. So, sometimes I get defects
in ruby-trunk.

@tenderlove said that Rails-3.2 works with
Ruby-2.0.
It means that many web applications can
work with Ruby-2.0. You can begin to use
Ruby-2.0 now.

Even though, you couldn’t run code with
Ruby-2.0 in production, you can run code
in test environment.
I think modern code have a lot of test
code. you can.

https://speakerdeck.com/kakutani/above-all-make-it-fun
If it was also difficult to run your code in
test environment, your hobby code should
run with Ruby-2.0.
I think that programmer have hobby code.
tDiary is hobby code. Matz said “Ruby is
hobby code!” too.

FFFFFFFFFFFFFFFFFFF.....F
FFFFFFFFFFFFF....F.F...FFFF
FFFFFFFFFFFFFFFFFFFFFF.
....FFFFFFFFFFFFFF....F.F...F
FFFFFFFFFFFFFFFFFFFFFF
FFF.....FFFFFFFFFFFFFF....F.
F...FFFFFFF.......FFFFFFFFFF
When you run hobby code with Ruby-2.0,
you will get error or segmentation fault.
You should report it ruby core team.

bugreport
* lib/open-uri.rb: use respond_to? to test Tempfile.
[ruby-dev:45995] [Bug #6781] reported by hsbt (Hiroshi SHIBATA).
* io.c (argf_next_argv): reset ARGF.next_p on ARGV.replace.
r34409 breaks replacing ARGV.
[ruby-dev:45160] [Bug #5952]
* ext/zlib/zlib.c (zstream_run_func): don't call inflate() when
z->stream.avail_in == 0. it return Z_BUF_ERROR.
but deflate() could be called with z->stream->avail_in == 0 because
it has hidden buffer in z->stream->state (opaque structure).
fix for gem install error. [ruby-dev:46149] [Bug #7040]
I discovered some defects in ruby-trunk.
One of these defects is ruby-trunk can’t
install some gems like libv8. Today’s ruby
is fixed by my reports.

A few weeks ago. I discovered a new
feature in cgi.rb. I thought this feature
broke backward compatibility. I reported it
for ruby-core.
In this result, this feature reverted.
You need to run your code with early
version of Ruby. and report defects,
behavior change and new feature.
If you detect behavior change after code
freeze, it’s too late.

Report
it!
You shouldn’t write these report in blog or
twitter. Probably, ruby-core commiters
don’t read your tweets.

Are commiter only people who develop
ruby? I don’t think so. this picture is one
of scene in rubyconf. matz, ko1 and JRuby
guys and many commiter and people
discussion “reﬁnements”
Your report and proposal of new feature
are making ruby!

Run your code
with 2.0.0
Let’s run your code with ruby 2.0.0

How to discover the Ruby's defects with web application

More Related Content

What's hot (20)

Viewers also liked (7)

Similar to How to discover the Ruby's defects with web application (20)

More from Hiroshi SHIBATA (20)

Recently uploaded (20)

How to discover the Ruby's defects with web application