Blogged by Ujihisa. Standard methods of programming and thoughts including Clojure, Vim, LLVM, Haskell, Ruby and Mathematics written by a Japanese programmer. github/ujihisa

Tuesday, May 26, 2009

Something Like Trinary Operator

Ruby has own trinary operator (conditional operator)

p true ? 1 : 2 #=> 1
p false ? 1 : 2 #=> 2

Let's make a something like it by Ruby.

class TrueClass
  def _?(&b)
    Trinary.new(b)
  end
end

class FalseClass
  def _?(&b)
    Trinary.new(nil)
  end
end

class Trinary
  def initialize(x)
    @x = x
  end

  def _(&y)
    @x ? @x.call : y.call
  end
end

p true._? { :a }._ { :b } #=> :a
p false._? { :a }._ { :b } #=> :b

The original idea was from nanki

Monday, May 25, 2009

Trying Merb

I tried My First Blog Tutorial which is documented in Merb official site.

Install

Via http://wiki.merbivore.com/installing_merb_from_edge

I'm not interested in any stable versions of software, so I decided to install edge merb.

$ git clone git://github.com/wycats/merb.git
$ cd merb
$ sudo rake install

I got messages include errors.

WARNING:  description and summary are identical
mv merb-1.1.gem pkg/merb-1.1.gem
Failed to install gem '/Users/ujihisa/git/merb/merb/pkg/merb-1.1.gem (1.1)' (merb requires merb_datamapper (= 1.1, runtime))
WARNING:  no rubyforge_project specified
WARNING:  description and summary are identical
mv merb-core-1.1.gem pkg/merb-core-1.1.gem
Successfully installed merb-core-1.1
Writing executable wrapper /opt/local/bin/merb
WARNING:  description and summary are identical
mv merb_datamapper-1.1.gem pkg/merb_datamapper-1.1.gem
Failed to install gem '/Users/ujihisa/git/merb/merb_datamapper/pkg/merb_datamapper-1.1.gem (1.1)' (merb_datamapper requires merb-actionorm (~> 1.1, runtime))
WARNING:  description and summary are identical
mv merb-action-args-1.1.gem pkg/merb-action-args-1.1.gem
Successfully installed merb-action-args-1.1
WARNING:  description and summary are identical
mv merb-assets-1.1.gem pkg/merb-assets-1.1.gem
Successfully installed merb-assets-1.1
WARNING:  description and summary are identical
mv merb-slices-1.1.gem pkg/merb-slices-1.1.gem
Successfully installed merb-slices-1.1
Writing executable wrapper /opt/local/bin/slice
WARNING:  description and summary are identical
mv merb-actionorm-1.1.gem pkg/merb-actionorm-1.1.gem
Successfully installed merb-actionorm-1.1
WARNING:  description and summary are identical
mv merb-auth-core-1.1.gem pkg/merb-auth-core-1.1.gem
WARNING:  description and summary are identical
mv merb-auth-more-1.1.gem pkg/merb-auth-more-1.1.gem
WARNING:  description and summary are identical
mv merb-auth-slice-password-1.1.gem pkg/merb-auth-slice-password-1.1.gem
WARNING:  description and summary are identical
mv merb-auth-1.1.gem pkg/merb-auth-1.1.gem
Successfully installed merb-auth-core-1.1
Successfully installed merb-auth-more-1.1
Successfully installed merb-auth-slice-password-1.1
Successfully installed merb-auth-1.1
WARNING:  description and summary are identical
mv merb-cache-1.1.gem pkg/merb-cache-1.1.gem
Successfully installed merb-cache-1.1
WARNING:  no rubyforge_project specified
WARNING:  description and summary are identical
mv merb-exceptions-1.1.gem pkg/merb-exceptions-1.1.gem
Successfully installed merb-exceptions-1.1
WARNING:  description and summary are identical
mv merb-gen-1.1.gem pkg/merb-gen-1.1.gem
Successfully installed merb-gen-1.1
Writing executable wrapper /opt/local/bin/merb-gen
WARNING:  description and summary are identical
mv merb-haml-1.1.gem pkg/merb-haml-1.1.gem
Successfully installed merb-haml-1.1
WARNING:  description and summary are identical
mv merb-helpers-1.1.gem pkg/merb-helpers-1.1.gem
Successfully installed merb-helpers-1.1
WARNING:  description and summary are identical
mv merb-mailer-1.1.gem pkg/merb-mailer-1.1.gem
Successfully installed merb-mailer-1.1
WARNING:  description and summary are identical
mv merb-param-protection-1.1.gem pkg/merb-param-protection-1.1.gem
Successfully installed merb-param-protection-1.1
ERROR:  could not find gem pkg/merb-more-1.1.gem locally or in a repository

I tried to go to next step without considering those errors.

$ cd merb/merb-core
$ sudo rake install
$ cd ../merb-auth
$ sudo rake install

They succeeded.

My First Blog

$ cd ~/tmp
$ merb-gen app my_first_merblog --template-engine=haml
$ cd my_first_blog
$ ls
Rakefile  app/      autotest/ config/   doc/      gems/     merb/ public/   spec/     tasks/

cool.

$ merb-gen resource post title:string,body:string
Loading init file from /Users/ujihisa/tmp/my_first_merblog/config/init.rb
Loading /Users/ujihisa/tmp/my_first_merblog/config/environments/development.rb
 ~
 ~ FATAL: The gem merb_datamapper (>= 0, runtime), [] was not found
 ~

Noooo! I don't know whether the last errors caused or not, but I tried below

$ cd ~/git/merb/merb_datamapper/
$ sudo rake install
$ cd -
$ merb-gen resource post title:string,body:string
Loading init file from /Users/ujihisa/tmp/my_first_merblog/config/init.rb
Loading /Users/ujihisa/tmp/my_first_merblog/config/environments/development.rb
Generating with resource generator:
Loading /Users/ujihisa/tmp/my_first_merblog/config/environments/development.rb
Loading /Users/ujihisa/tmp/my_first_merblog/config/environments/development.rb
Loading /Users/ujihisa/tmp/my_first_merblog/config/environments/development.rb
     [ADDED]  spec/models/post_spec.rb
     [ADDED]  app/models/post.rb
     [ADDED]  spec/requests/posts_spec.rb
     [ADDED]  app/views/posts/show.html.haml
     [ADDED]  app/views/posts/index.html.haml
     [ADDED]  app/views/posts/edit.html.haml
     [ADDED]  app/views/posts/new.html.haml
     [ADDED]  app/controllers/posts.rb
     [ADDED]  app/helpers/posts_helper.rb
resources :posts route added to config/router.rb

Yay!

Contrary to rails, specs has been generated instead of tests. Let's spec!

$ rake spec
(in /Users/ujihisa/tmp/my_first_merblog)
Loading init file from /Users/ujihisa/tmp/my_first_merblog/config/init.rb
Loading /Users/ujihisa/tmp/my_first_merblog/config/environments/development.rb
./spec/models/../spec_helper.rb:17: uninitialized constant Merb::Test::ViewHelper (NameError)
        from /opt/local/lib/ruby/gems/1.8/gems/rspec-1.2.6/lib/spec/runner.rb:41:in `configure'
        from ./spec/models/../spec_helper.rb:16
        from ./spec/models/post_spec.rb:1:in `require'
        from ./spec/models/post_spec.rb:1
        from /opt/local/lib/ruby/gems/1.8/gems/rspec-1.2.6/lib/spec/runner /example_group_runner.rb:15:in `load' 
        from /opt/local/lib/ruby/gems/1.8/gems/rspec-1.2.6/lib/spec/runner /example_group_runner.rb:15:in `load_files' 
        from /opt/local/lib/ruby/gems/1.8/gems/rspec-1.2.6/lib/spec/runner /example_group_runner.rb:14:in `each' 
        from /opt/local/lib/ruby/gems/1.8/gems/rspec-1.2.6/lib/spec/runner /example_group_runner.rb:14:in `load_files' 
        from /opt/local/lib/ruby/gems/1.8/gems/rspec-1.2.6/lib/spec/runner/options.rb:99:in `run_examples'

        from /opt/local/lib/ruby/gems/1.8/gems/rspec-1.2.6/lib/spec/runner /command_line.rb:9:in `run' 
        from /opt/local/lib/ruby/gems/1.8/gems/rspec-1.2.6/bin/spec:4
    rake aborted!
    Command /opt/local/bin/ruby
-I"/opt/local/lib/ruby/gems/1.8/gems/rspec-1.2.6/lib"
"/opt/local/lib/ruby/gems/1.8/gems/rspec-1.2.6/bin/spec"
"spec/models/post_spec.rb" "spec/requests/posts_spec.rb" --options
./spec/spec.opts failed


(See full trace by running task with --trace)

Hmm... There seem to be a lot of works to do.

Snap out of it. In accordance with the instruction of the tutorial, I did the following command,

$ merb-gen resource comment post_id:integer,name:string,body:string

and I fixed config/router.rb.

diff --git a/config/router.rb b/config/router.rb
index 173d73c..360618d 100644
--- a/config/router.rb
+++ b/config/router.rb
@@ -27,8 +27,9 @@

 Merb.logger.info("Compiling routes...")
 Merb::Router.prepare do
-  resources :comments
-  resources :posts
+  resources :posts do |p|
+    p.resources :comments
+  end
   # RESTful routes
   # resources :posts

@@ -43,4 +44,4 @@ Merb::Router.prepare do

   # Change this for your home page to be available at /
   # match('/').to(:controller => 'whatever', :action =>'index')
-end
\ No newline at end of file
+end

Then I set up the relationships, controller and view. This diff is very long, so I posted the diff to gist.

http://gist.github.com/117793

Let's run!

$ rake db:automigrate
$ merb
$ open http://localhost:4000

merb

merb2

Cool!

But when I posted a comment to a post, an error occured.

merb3

Something is wrong. Action '1' was not found in Posts... At first I suspected that config/routes.rb was wrong, but finally I found the cause is in app/views/comments/_comment.html.haml. Following patch fixes the problem.

diff --git a/app/views/comments/_comment.html.haml b/app/views/comments/_comment.html.haml 
index 7e54ce8..fcebb73 100644
--- a/app/views/comments/_comment.html.haml
+++ b/app/views/comments/_comment.html.haml
@@ -1,6 +1,6 @@
 %p= error_messages_for :comment

-= form_for(:comment, :action => "/posts/#{@post.id}/comments") do
+= form_for(:comment, :action => "/posts/#{@post.id}/comments", :method => :post) do 
   %p= text_field :name, :name => "comment[name]"
   %p= text_area  :body
   %p= submit "Comment"

The cause was the HTTP METHOD was put for some reason instead of desired post. By specifying post explicitly, the problem was solved.

I fixed the turorial wiki, so a person who tries it will never meet the problem aftertime.

Deploy

I deployed the sample merb application.

http://67.23.4.19:20090525/posts

The post number 20090525 is today's date.

This sample app server is already stopped.

Sunday, May 24, 2009

My Excerpt From C/C++ Support Vim Script

Today I read the document of c.vim. Features that I'll use later are outlined below.

Inserting a template/snippet

Key mappings to insert a template/snippet bellow.

Legend:  (i) insert mode, (n) normal mode, (v) visual mode

\c*       code -> comment /* */               (n,v)
\c/       code -> comment //                  (n,v)
\cc       code -> comment //                  (n,v)
\co       comment -> code                     (n,v)
\p<       #include <>                         (n,i)
\p"       #include ""                         (n,i)

I wonder why The first four key don't have operator-pending mode? We can never commentout 3 lines without using visual mode.

There are a lot of key mappings to insert a template of a snippet. I'm doubtful about the effectiveness of those snippets. The most surprising one is below:

\i0       for( x=0; x<n; x+=1 )               (n,v,i)

Of course we can easily fix those templates. see ~/.vim/c-support/codesnippets/. For example, \p< doesn't insert #include <> but #include\t<>, and even worse the snippet will inserted on the next line, therefore I have to fix the feature or to decide not to use it.

Syntax based folding

In the document, it is written that

Syntax based folding can be enabled by adding the following lines to the file '~/.vim/syntax/c.vim':

 syn region cBlock start="{" end="}" transparent fold
 set foldmethod=syntax
 " initially all folds open:
 set foldlevel=999

But I found that it does not work well. It has a lot of bug. Following setting works well:

" in ~/.vim/syntax/c.vim
set foldmethod=syntax

But I prefer that only functions are folded, so finally I deleted the file. I have to find how to do it.

Compile and Run

\rc       save and compile                    (n)
\rr       run                                 (n)
\ra       set comand line arguments           (n)
\rm       run make                            (n)
\rg       cmd. line arg. for make             (n)
\rp       run splint                          (n)

I don't understand of there is a special key mapping \rc or \rm instead of using preexistent :make feature. I checked :compiler but it was not set.

I always set <space>m as :make, so I wrote below on my vimrc.

augroup MyCompiler
  autocmd!
  autocmd Filetype c nmap <buffer> <Space>m \rc
  autocmd Filetype cpp nmap <buffer> <Space>m \rc
augroup END

\rr is OK. I can use both \rr feature which outputs temporary and quickrun.vim which outputs permanentry.

Thursday, May 21, 2009

Vancouver.rb: Rubish and Erlang

I attended the Vancouver.rb meeting: Rubish with Howard Yeh AND Intro to Erlang with Ken Pratt. My notes are below -- while it doesn't make sense without the original talks.

Rubish

  • Closes the gap between script and shell.
  • Extensible is a very important feature of Rubish
  • Executable {Cmd, Pipe}, Job control, Context

    (sh) ls a b c
    (rb) ls 'a b c'
    
    (sh) ls 'a b c'
    (rb) ls.q 'a b c'
    

Sed and Awk

  • sed: one liner. Rubish has sed method.

    (rb) find('. -name "*.el*').sed { p if File.ex... }.map {|f| ...
    
  • Be explict, fits well

  • Optimization and consistency

  • Grep

    p prev if respond_to?(:prev)
    

Aggregator

  • inspired by Common Lisp
  • collect (not Enumerable#collect)

Concurrency

  • No interaction between tasks. No deadlock, no shared memory, but is safety by policy.
  • Background jobs: Executable#{exec!, each! map!}, Job#{wait}

Context

    obj.extend(module).instance_eval { ... }

with method

Intro to Erlang

  • <3 asynchronous message passing
  • Processes shoud be able to run forever
  • <3 tails recursion
  • Compiled to bytecode, runs on VM
  • Libraries
    • OTP
    • Supervisor trees: all nodes supervise others(?)
    • Mnesia distributed DB, Web frameworks Nitrogen, Webmachine
    • JSON
  • Amazon SimpleDB
  • twitter: ejabberd
  • 37signals: Campfire
  • github
  • Ruby + Erlang (Railsconf keynote) (Haskell)
  • Katamari/Fuzed
    • Rails server clustering
    • Erlang bindings for Rails
    • YAWS web server
  • Nanite
    • A self-assebing cluster of Ruby daemons
    • rails server or map/reduce tasks
    • RabbitMQ handles the distributed stuff
  • Erlectricity: erlang/ruby interaction
    • Bi-directional communication using Erlang's binary fomat
    • Erlang treats Ruby processes as just anohter erlang process
  • http://www.eralng-factory.com/
  • Erlang meetup at vancouver
  • Haskell is an academic language. more purely
    • they don't want to be popular!
    • erlang is not lazy evaluation
  • debugging
    • GUI tool, breakpoints, multi processes

My impression

  • It was hard for me to listen to English talks, even in tech talks.
  • I got interested in the implementation of Rubish.

Monday, May 18, 2009

Pseudo URI Literal

A URI notation is easy to be distinguished. There is almost no confusion about it. Therefore it may be OK that some programming languages have URI Literal. Unfortunately, I have never seen.

I added a something like a URI literal into Ruby.

def http(x)
  "http://#{x}"
end

class Symbol
  def /(o)
    o
  end
end

class String
  def method_missing(*a, &b)
    raise unless a.size == 1
    "#{self}.#{a.first}"
  end

  def /(o)
    "#{self}/#{o}"
  end
end

def method_missing(*a, &b)
  raise unless a.size == 1
  a.first.to_s
end

p http://www.google.com/this/is/a/pen.html
#=> "http://www.google.com/this/is/a/pen.html"

Although this snippet supports only http, it is easy to be extended to handle https, ftp or gopher.

Added on 2009-05-20

I fixed the code to support % notation and to return a URI object by a URI literal as _tad_ suggested. The new code is below:

require 'uri'

def http(x)
  URI.parse "http://#{x}"
end

class Symbol
  def /(o)
    o
  end
end

class String
  def method_missing(*a, &b)
    raise unless a.size == 1
    "#{self}.#{a.first}"
  end

  def /(o)
    "#{self}/#{o}"
  end

  def %(n)
    "#{self}%#{n}"
  end
end

class Integer
  def method_missing(*a, &b)
    raise unless a.size == 1
    "#{self}.#{a.first}"
  end
end

def method_missing(*a, &b)
  raise unless a.size == 1
  a.first.to_s
end

p http://www.google.com/this/is/a/pen.html
#=> #<URI::HTTP:0x7a8c8 URL:http://www.google.com/this/is/a/pen.html>

p http://www.google.com/this/is/a/aaa%20.html
#=> #<URI::HTTP:0x79c98 URL:http://www.google.com/this/is/a/aaa%20.html>

To tell the truth, I finished writing the code in a short time, but at that time blogger.vim didn't work. It tool very long time to fix blogger.vim, and then finally I succeeded it. blogger.vim will be version up soon.

Visited SFU: Simon Fraser University

Today I visited SFU Burnaby campus.

Heron

It took an hour to get to SFU. I had to transfer 2 times. The bus from the nearest station, the Production Way/University Station, to the campus was absolutely congested with students. I remembered the terrible commuter bus of my high school in Japan.

Bus

Meanwhile the campus was cool. There was something like relic atmosphere. I remembered the Japanese famous game Shadow of the Colossus.

Cafeteria

I took an espresso in a cafeteria and then I took a class about the Human Interface.

Path

Garden

And then I looked around mainly Computing Science department. I wanted to talk with people there but I was too nervous to do it.

Campus map

Reflection

After that I went to a library and read some journals about programming languages.

Library

Library2

Blogger.vim 1.0 Released

I wrote a Vim script blogger.vim which handles blogger (blogspot) by Vim. This entry is powered by the baby blogger.vim. Blogger.vim is the only one vim script which handles Blogger using metarw.

Screenshot of blogger.vim

I wrote a ruby script blogger.rb and used it for blogger.vim. Blogger.rb is separated from blogger.vim, so you don't need if_ruby option for your vim.

Blogger.vim 1.0 has a lot of known bugs and unknown bugs. If you find a bug, please let me know it after reading the known bugs section of README.md.

enjoy!

Thursday, May 14, 2009

Memory exhausted in a deep nested array literal

When I was reading '[' section of parser.y,

7190       case '[':
7191         paren_nest++;

I got interested in how it will happen when paren_nest becomes huge. I tried this script:

(9995..11000).each do |n|
  p n
  eval '['*n + ']'*n
end

The result was

/var/folders/Dz/Dz5WpFSZGUaFLA8jp8kT5E+++TM/-Tmp-/v258465/130:3: (eval):1: compile error (SyntaxError)
(eval):1: memory exhausted
...[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]...
                              ^
  from /var/folders/Dz/Dz5WpFSZGUaFLA8jp8kT5E+++TM/-Tmp-/v258465/130:1:in `eval'
  from /var/folders/Dz/Dz5WpFSZGUaFLA8jp8kT5E+++TM/-Tmp-/v258465/130:3
  from /var/folders/Dz/Dz5WpFSZGUaFLA8jp8kT5E+++TM/-Tmp-/v258465/130:1:in `each'
  from /var/folders/Dz/Dz5WpFSZGUaFLA8jp8kT5E+++TM/-Tmp-/v258465/130:1

hmm

Wednesday, May 13, 2009

Fixed uri-open of Termtter

Termtter's uri-open plugin

I committed some to Termtter. The following diff is the summary of my patches today.

diff --git a/lib/plugins/uri-open.rb b/lib/plugins/uri-open.rb
index 0afe73f..06a521b 100644
--- a/lib/plugins/uri-open.rb
+++ b/lib/plugins/uri-open.rb
@@ -8,44 +8,43 @@ module Termtter::Client
     :points => [:output],
     :exec_proc => lambda {|statuses, event|
       statuses.each do |s|
-        public_storage[:uris] += s[:text].scan(%r|https?://[^\s]+|)
+        public_storage[:uris] = s[:text].scan(%r|https?://[^\s]+|) + public_storage[:uris]
       end
     }
   )

   def self.open_uri(uri)
-    unless config.plugins.uri_open.browser.empty?
-      system config.plugins.uri_open.browser, uri
-    else
-      case RUBY_PLATFORM
-      when /linux/
-        system 'firefox', uri
-      when /mswin(?!ce)|mingw|bccwin/
-        system 'explorer', uri
+    cmd =
+      unless config.plugins.uri_open.browser.empty?
+        config.plugins.uri_open.browser
       else
-        system 'open', uri
+        case RUBY_PLATFORM
+        when /linux/; 'firefox'
+        when /mswin(?!ce)|mingw|bccwin/; 'explorer'
+        else; 'open'
+        end
       end
-    end
+    system cmd, uri
   end

   register_command(
     :name => :'uri-open', :aliases => [:uo],
-    :exec_proc => lambda{|arg|
+    :exec_proc => lambda {|arg|
       case arg
-      when /^\s+$/
-        public_storage[:uris].each do |uri|
-          open_uri(uri)
-        end
-        public_storage[:uris].clear
+      when ''
+        open_uri public_storage[:uris].shift
       when /^\s*all\s*$/
-        public_storage[:uris].each do |uri|
-          open_uri(uri)
-        end
-        public_storage[:uris].clear
+        public_storage[:uris].
+          each {|uri| open_uri(uri) }.
+          clear
       when /^\s*list\s*$/
-        public_storage[:uris].each_with_index do |uri, index|
-          puts "#{index}: #{uri}"
-        end
+        public_storage[:uris].
+          enum_for(:each_with_index).
+          to_a.
+          reverse.
+          each  do |uri, index|
+            puts "#{index}: #{uri}"
+          end
       when /^\s*delete\s+(\d+)\s*$/
         puts 'delete'
         public_storage[:uris].delete_at($1.to_i)
@@ -53,19 +52,20 @@ module Termtter::Client
         public_storage[:uris].clear
         puts "clear uris"
       when /^\s*(\d+)\s*$/
-        open_uri(public_storage[:uris][$1.to_i])
-        public_storage[:uris].delete_at($1.to_i)
+        open_uri(public_storage[:uris].delete_at($1.to_i))
+      else
+        puts "**parse error in uri-open**"
       end
     },
-    :completion_proc => lambda{|cmd, arg|
-      %w(all list delete clear).grep(/^#{Regexp.quote arg}/).map{|a| "#{cmd} #{a}"}
+    :completion_proc => lambda {|cmd, arg|
+      %w(all list delete clear).grep(/^#{Regexp.quote arg}/).map {|a| "#{cmd} #{a}" }
     }
   )
 end
-# ~/.termtter
+# ~/.termtter/config
 # plugin 'uri-open'
 #
 # see also: http://ujihisa.nowa.jp/entry/c3dd00c4e0
 #
 # KNOWN BUG
-# * In Debian, exit or C-c in the termtter kills your firefox.
+# * In Debian, exit or C-c in the termtter would kill your firefox.

Lisp like symbol in Ruby

Lisp like symbol in Ruby

I succeeded in adding a lisp like symbol literal to ruby by fixing parse.y.

diff --git a/parse.y b/parse.y
index e2e92ce..c9fbb03 100644
--- a/parse.y
+++ b/parse.y
@@ -6319,6 +6319,9 @@ static int
 parser_yylex(struct parser_params *parser)
 {
     register int c;
+    int cs[1024];
+    int i;
+    char flag; // 't': sTring, 'y': sYmbol
     int space_seen = 0;
     int cmd_state;
     enum lex_state_e last_state;
@@ -6631,8 +6634,26 @@ parser_yylex(struct parser_params *parser)
         return tXSTRING_BEG;

       case '\'':
-        lex_strterm = NEW_STRTERM(str_squote, '\'', 0);
-        return tSTRING_BEG;
+        flag = 'y'; // sYmbol by default
+        for (i=0; i<sizeof(cs); i++) {
+            cs[i] = nextc();
+            if (cs[i] == '\'') {
+                flag = 't'; // sTring
+                break;
+            }
+            if (cs[i] == '\n' || cs[i] == -1) {
+                break; // sYmbol
+            }
+        }
+        while (i >= 0)
+            pushback(cs[i--]);
+        if (flag == 't') {
+            lex_strterm = NEW_STRTERM(str_squote, '\'', 0);
+            return tSTRING_BEG;
+        } else {
+            lex_state = EXPR_FNAME;
+            return tSYMBEG;
+        }

       case '?':
         if (lex_state == EXPR_END || lex_state == EXPR_ENDARG) {

After applying this patch, we can write such like:

p 'aaa'      #=> "aaa"
p 'bbb       #=> :bbb
p 'ccc, true #=> :ccc
             #   true
p :ddd       #=> :ddd

The followings are the methodology how it works:

  1. If the ruby parser finds single quote, the parser peeks the next characters from the next character
  2. The parser stocks each characters into a stack which maximum size is 1024
  3. If the parser finds another single quote, the parser does back tracking with the stack and then considers the following characters as a string.
  4. Otherwise if the parser does not find another single quote within the line, the parser does back tracking with the stack and then considers the following characters as a symbol.

Obviously, there are some known bugs:

  • It cannot handle a line which has 1024+ characters after a quote
  • It confuses in p 'aaa, 'bbb
  • It cannot handle a multi line string

e.g.

p 'aaa
   bbb'

To tell the truth, I wrote this patch just for my curiousity. I don't believe this syntax will harmonize well with Ruby.

Tuesday, May 12, 2009

Cryptic infinite recursion in parse.y

When a single quote comes, parse.y will do:

case '\'':
  lex_strterm = NEW_STRTERM(str_squote, '\'', 0);
  return tSTRING_BEG;

This piece of codes is very same as a part of :'aaa'

NEW_STRTERM is a macro. It is a wrapper of a macro rb_node_newnode, which is a wrapper of a function node_newnode

#define NEW_STRTERM(func, term, paren) \
        rb_node_newnode(NODE_STRTERM, (func), (term) | ((paren) << (CHAR_BIT * 2)), 0)

#define rb_node_newnode(type, a1, a2, a3) node_newnode(parser, type, a1, a2, a3)

static NODE*
node_newnode(struct parser_params *parser, enum node_type type, VALUE a0, VALUE a1, VALUE a2)
{
    NODE *n = (rb_node_newnode)(type, a0, a1, a2);
    nd_set_line(n, ruby_sourceline);
    return n;
}

The last function node_newnode calls rb_node_newnode, that is node_newnode itself, recursively.

Here is the node_newnode which macro is expanded:

static NODE*
node_newnode(struct parser_params *parser, enum node_type type, VALUE a0, VALUE a1, VALUE a2)
{
    NODE *n = node_newnode(parser, type, a0, a1, a2);
    nd_set_line(n, ruby_sourceline);
    return n;
}

It's mysterious. node_newnode seems infinite recursion without any terminate conditions. Why does it work?

Read parse.y and learn about a colon

In Ruby, there are a lot of usages of colon.

  • Symbol :aaa
  • Symbol with single quotes :'aaa'
  • Symbol with double quotes :"aaa"
  • Conditional operator aaa ? bbb : ccc
  • Class hierarchy A::B

The following is an excerpt from parse.y relates a colon:

case ':':
  c = nextc();
  if (c == ':') {
      if (IS_BEG() ||
          lex_state == EXPR_CLASS || (IS_ARG() && space_seen)) {
          lex_state = EXPR_BEG;
          return tCOLON3;
      }
      lex_state = EXPR_DOT;
      return tCOLON2;
  }
  if (lex_state == EXPR_END || lex_state == EXPR_ENDARG || (c != -1 && ISSPACE(c))) {
      pushback(c);
      lex_state = EXPR_BEG;
      return ':';
  }
  switch (c) {
    case '\'':
      lex_strterm = NEW_STRTERM(str_ssym, c, 0);
      break;
    case '"':
      lex_strterm = NEW_STRTERM(str_dsym, c, 0);
      break;
    default:
      pushback(c);
      break;
  }
  lex_state = EXPR_FNAME;
  return tSYMBEG;

If the next character of ':' is ':', in a nutshell if '::' has come, it returns tCOLON3 (:: at the very left) or tCOLON2 (:: at the right hand). Otherwise if the colon is just after expressions, it's a conditional operator. Ditto if the next character is a space. Otherwise finally the colon is a prefix of symbol.

I've never known that we can have a newline after ::!

class A::

B
end

It's valid.

OK... So, let's try to add a new literal :-) which has equivalent to =>.

diff --git a/parse.y b/parse.y
index e2e92ce..8e49bf4 100644
--- a/parse.y
+++ b/parse.y
@@ -7082,6 +7082,9 @@ parser_yylex(struct parser_params *parser)

       case ':':
         c = nextc();
+        if (c == '-' && nextc() == ')') {
+            return tASSOC;
+        }
         if (c == ':') {
             if (IS_BEG() ||
                 lex_state == EXPR_CLASS || (IS_ARG() && space_seen)) {

That's easy.

$ ./ruby -e 'p({ 1 :-) 2 })'
{1=>2}

cool!

Monday, May 11, 2009

Save the iTerm

System Preferences

Now you don't have to care about typing Cmd-Q or Cmd-W by mistake.

Thursday, May 7, 2009

About Array#<=> (a.k.a. Spacecraft Operator)

The documentations about Array#<=> are:

For example:

[1, 2, 3, 4] <=> [1, 2, 3, 3] #=> 1
[1, 2, 3, 4] <=> [1, 2, 3, 4] #=> 0
[1, 2, 3, 4] <=> [1, 2, 3, 5] #=> -1
[1, 2, 3, 4] <=> [1, 2, 3] #=> 1

The implementation in MRI 1.9 is:

VALUE
rb_ary_cmp(VALUE ary1, VALUE ary2)
{
    long len;
    VALUE v;

    ary2 = to_ary(ary2);
    if (ary1 == ary2) return INT2FIX(0);
    v = rb_exec_recursive(recursive_cmp, ary1, ary2);
    if (v != Qundef) return v;
    len = RARRAY_LEN(ary1) - RARRAY_LEN(ary2);
    if (len == 0) return INT2FIX(0);
    if (len > 0) return INT2FIX(1);
    return INT2FIX(-1);
}

static VALUE
recursive_cmp(VALUE ary1, VALUE ary2, int recur)
{
    long i, len;

    if (recur) return Qnil;
    len = RARRAY_LEN(ary1);
    if (len > RARRAY_LEN(ary2)) {
        len = RARRAY_LEN(ary2);
    }
    for (i=0; i<len; i++) {
        VALUE v = rb_funcall(rb_ary_elt(ary1, i), id_cmp, 1, rb_ary_elt(ary2, i));
        if (v != INT2FIX(0)) {
            return v;
        }
    }
    return Qundef;
}

I translated it from C to Ruby literally:

class Array
  def yet_another_cmp(you)
    you = you.to_ary
    return 0 if self == you
    v = nil
    (0...[self.length, you.length].min).each do |i|
      v = self[i] <=> you[i]
      return v if v != 0
    end
    len = self.length - you.length
    return 0 if len == 0
    return 1 if len > 0
    -1
  end
end

It works exactly the same.

You may think it is easy to write a simpler equivalent implement with Array#zip, but unfortunately I found that it was not so simple. The following is the simplest code I can write. Of course it works exactly the same as original <=>.

class Array
  def simple_cmp(you)
    self.zip(you) {|x, y|
      break if (x && y).nil?
      (v = x <=> y) == 0 or return v
    }
    self.length <=> you.length
  end
end

In conclusion, Array#<=> itself is complicated one. Enjoy your space travel!

Saturday, May 2, 2009

A benchmark of Array#to_s

A bugfix patch may make Array#to_s slow. I am finding out how slow it becomes.

That patch changes the definition of Array#to_s from:

rb_define_method(rb_cArray, "to_s", rb_ary_inspect, 0);
rb_define_method(rb_cArray, "inspect", rb_ary_inspect, 0);

to:

rb_define_method(rb_cArray, "inspect", rb_ary_inspect, 0);
rb_define_alias(rb_cArray,  "to_s", "inspect");

This change effects only in the array has itself recursively.

Benchmark code I use is:

require 'benchmark'
short_a = Array.new(100) { rand }
long_a = Array.new(10000) { rand }

Benchmark.bmbm do |b|
  b.report do
    1000.times do
      short_a.to_s
    end
  end

  b.report do
    10.times do
      long_a.to_s
    end
  end
end

And the results are below.

Conventional (before the patch applied)

Rehearsal ------------------------------------
   0.810000   0.010000   0.820000 (  0.836625)
   0.870000   0.010000   0.880000 (  0.901661)
--------------------------- total: 1.700000sec

       user     system      total        real
   0.740000   0.010000   0.750000 (  0.748501)
   0.770000   0.010000   0.780000 (  0.801869)

Current (after the patch applied)

Rehearsal ------------------------------------
   0.820000   0.000000   0.820000 (  0.852360)
   0.860000   0.010000   0.870000 (  0.897602)
--------------------------- total: 1.690000sec

       user     system      total        real
   0.850000   0.010000   0.860000 (  0.873125)
   0.830000   0.010000   0.840000 (  0.871903)

There are certainly slow changes, but they seem enough slight. What do you think about it?

Followers