[Groonga-commit] ranguba/chupa-text at aff7fbf [master] Add CSV decomposer

Back to archive index

Kouhei Sutou null+****@clear*****
Sat Jan 4 20:38:30 JST 2014


Kouhei Sutou	2014-01-04 20:38:30 +0900 (Sat, 04 Jan 2014)

  New Revision: aff7fbf6947be061625e469a40cf1f4c815a98de
  https://github.com/ranguba/chupa-text/commit/aff7fbf6947be061625e469a40cf1f4c815a98de

  Message:
    Add CSV decomposer

  Added files:
    lib/chupa-text/decomposers/csv.rb
    test/decomposers/test-csv.rb
  Modified files:
    data/mime-types.conf

  Modified: data/mime-types.conf (+3 -0)
===================================================================
--- data/mime-types.conf    2014-01-04 20:31:55 +0900 (fa4af83)
+++ data/mime-types.conf    2014-01-04 20:38:30 +0900 (415315f)
@@ -14,3 +14,6 @@ mime_type["xhtml"] = "application/xhtml+xml"
 mime_type["xml"] = "text/xml"
 
 mime_type["css"] = "text/css"
+
+mime_type["csv"] = "text/csv"
+mime_type["tsv"] = "text/tab-separated-values"

  Added: lib/chupa-text/decomposers/csv.rb (+44 -0) 100644
===================================================================
--- /dev/null
+++ lib/chupa-text/decomposers/csv.rb    2014-01-04 20:38:30 +0900 (67a81c8)
@@ -0,0 +1,44 @@
+# Copyright (C) 2013  Kouhei Sutou <kou �� clear-code.com>
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
+
+require "csv"
+
+module ChupaText
+  module Decomposers
+    class CSV < Decomposer
+      registry.register("csv", self)
+
+      def target?(data)
+        data.extension == "csv" or
+          data.mime_type == "text/csv"
+      end
+
+      def decompose(data)
+        text = ""
+        data.open do |input|
+          csv = ::CSV.new(input)
+          csv.each do |row|
+            text << row.join(" ")
+            text << "\n"
+          end
+        end
+        text_data = TextData.new(text)
+        text_data.uri = data.uri
+        yield(text_data)
+      end
+    end
+  end
+end

  Added: test/decomposers/test-csv.rb (+48 -0) 100644
===================================================================
--- /dev/null
+++ test/decomposers/test-csv.rb    2014-01-04 20:38:30 +0900 (3fb6455)
@@ -0,0 +1,48 @@
+# Copyright (C) 2013  Kouhei Sutou <kou �� clear-code.com>
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
+
+class TestDecomposersCSV< Test::Unit::TestCase
+  include Helper
+
+  def setup
+    @decomposer = ChupaText::Decomposers::CSV.new({})
+  end
+
+  sub_test_case("decompose") do
+    def test_body
+      csv = <<-CSV
+Hello,World
+Ruby,ChupaText
+      CSV
+      assert_equal([csv.gsub(/,/, " ")],
+                   decompose(csv).collect(&:body))
+    end
+
+    private
+    def decompose(csv)
+      data = ChupaText::Data.new
+      data.path = "hello.csv"
+      data.mime_type = "text/csv"
+      data.body = csv
+
+      decomposed = []
+      @decomposer.decompose(data) do |decomposed_data|
+        decomposed << decomposed_data
+      end
+      decomposed
+    end
+  end
+end
-------------- next part --------------
HTML����������������������������...
Download 



More information about the Groonga-commit mailing list
Back to archive index