OSDN -- Develop and Download Open Source Software
  • R/O
  • SSH
  • HTTPS

Commit

Frequently used words (click to add to your profile)

javac++androidlinuxc#windowsobjective-ccocoa誰得qtpythonphprubygameguibathyscaphec計画中(planning stage)翻訳omegatframeworktwitterdomtestvb.netdirectxゲームエンジンbtronarduinopreviewer

作業部屋の使い方を試しています。


Commit MetaInfo

Revision134 (tree)
Time2015-03-31 11:57:06
Authortuna_p

Log Message

ブランチ作成

Change Summary

Incremental Difference

--- branches/b4/webScraping/data/Yahoo!天気.xml (nonexistent)
+++ branches/b4/webScraping/data/Yahoo!天気.xml (revision 134)
@@ -0,0 +1,71 @@
1+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
2+<xmlcontainer>
3+<webscraping>
4+<url>http://weather.yahoo.co.jp/weather/</url>
5+<searchlist listNo="1">
6+<item>天気０１</item>
7+<htmltag>li</htmltag>
8+<htmlclass>point pt1400</htmlclass>
9+</searchlist>
10+<searchlist listNo="2">
11+<item>天気０２</item>
12+<htmltag>li</htmltag>
13+<htmlclass>point pt1900</htmlclass>
14+</searchlist>
15+<searchlist listNo="3">
16+<item>天気０３</item>
17+<htmltag>li</htmltag>
18+<htmlclass>point pt3410</htmlclass>
19+</searchlist>
20+<searchlist listNo="4">
21+<item>天気０４</item>
22+<htmltag>li</htmltag>
23+<htmlclass>point pt4410</htmlclass>
24+</searchlist>
25+<searchlist listNo="5">
26+<item>天気０５</item>
27+<htmltag>li</htmltag>
28+<htmlclass>point pt5110</htmlclass>
29+</searchlist>
30+<searchlist listNo="6">
31+<item>天気０６</item>
32+<htmltag>li</htmltag>
33+<htmlclass>point pt5410</htmlclass>
34+</searchlist>
35+<searchlist listNo="7">
36+<item>天気０７</item>
37+<htmltag>li</htmltag>
38+<htmlclass>point pt5610</htmlclass>
39+</searchlist>
40+<searchlist listNo="8">
41+<item>天気０８</item>
42+<htmltag>li</htmltag>
43+<htmlclass>point pt6200</htmlclass>
44+</searchlist>
45+<searchlist listNo="9">
46+<item>天気０９</item>
47+<htmltag>li</htmltag>
48+<htmlclass>point pt6710</htmlclass>
49+</searchlist>
50+<searchlist listNo="10">
51+<item>天気１０</item>
52+<htmltag>li</htmltag>
53+<htmlclass>point pt7410</htmlclass>
54+</searchlist>
55+<searchlist listNo="11">
56+<item>天気１１</item>
57+<htmltag>li</htmltag>
58+<htmlclass>point pt8210</htmlclass>
59+</searchlist>
60+<searchlist listNo="12">
61+<item>天気１２</item>
62+<htmltag>li</htmltag>
63+<htmlclass>point pt8810</htmlclass>
64+</searchlist>
65+<searchlist listNo="13">
66+<item>天気１３</item>
67+<htmltag>li</htmltag>
68+<htmlclass>point pt9110</htmlclass>
69+</searchlist>
70+</webscraping>
71+</xmlcontainer>
--- branches/b4/webScraping/data/Yahoo!ファイナンス.xml (nonexistent)
+++ branches/b4/webScraping/data/Yahoo!ファイナンス.xml (revision 134)
@@ -0,0 +1,217 @@
1+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
2+<xmlcontainer>
3+<webscraping>
4+<url>http://stocks.finance.yahoo.co.jp/stocks/detail/?code=5020.T</url>
5+<searchlist listNo="1">
6+<item>銘柄コード</item>
7+<htmltag>dl</htmltag>
8+<htmlclass>stocksInfo clearFix</htmlclass>
9+<regexp>(^\d{4})</regexp>
10+</searchlist>
11+<searchlist listNo="2">
12+<item>カテゴリ</item>
13+<htmltag>div</htmltag>
14+<htmlclass>stockMainTabParts stockMainTabPartsCurrent</htmlclass>
15+</searchlist>
16+<searchlist listNo="3">
17+<item>業種</item>
18+<htmltag>dd</htmltag>
19+<htmlclass>category yjSb</htmlclass>
20+</searchlist>
21+<searchlist listNo="4">
22+<item>取得時間</item>
23+<htmltag>dd</htmltag>
24+<htmlclass>yjSb real</htmlclass>
25+<regexp>^(.*)\t</regexp>
26+</searchlist>
27+<searchlist listNo="5">
28+<item>銘柄名</item>
29+<htmltag>th</htmltag>
30+<htmlclass>symbol</htmlclass>
31+</searchlist>
32+<searchlist listNo="6">
33+<item>株価</item>
34+<htmltag>td</htmltag>
35+<htmlclass>stoksPrice</htmlclass>
36+</searchlist>
37+<searchlist listNo="7">
38+<item>前日比</item>
39+<htmltag>td</htmltag>
40+<htmlclass>change</htmlclass>
41+<regexp>\t(.*)（.*%）</regexp>
42+</searchlist>
43+<searchlist listNo="8">
44+<item>前日比％</item>
45+<htmltag>td</htmltag>
46+<htmlclass>change</htmlclass>
47+<regexp>\t.*（(.*)%）</regexp>
48+</searchlist>
49+<searchlist listNo="9">
50+<item>前日終値</item>
51+<htmltag>div</htmltag>
52+<htmlclass>lineFi clearfix</htmlclass>
53+<around>0</around>
54+<regexp>^([,.0-9]+)\t</regexp>
55+</searchlist>
56+<searchlist listNo="10">
57+<item>始値</item>
58+<htmltag>div</htmltag>
59+<htmlclass>lineFi clearfix</htmlclass>
60+<around>1</around>
61+<regexp>^([,.0-9]+|-{3})\t</regexp>
62+</searchlist>
63+<searchlist listNo="11">
64+<item>高値</item>
65+<htmltag>div</htmltag>
66+<htmlclass>lineFi clearfix</htmlclass>
67+<around>2</around>
68+<regexp>^((ストップ高\t|ストップ安\t)?[,0-9]+|-{3})</regexp>
69+</searchlist>
70+<searchlist listNo="12">
71+<item>安値</item>
72+<htmltag>div</htmltag>
73+<htmlclass>lineFi clearfix</htmlclass>
74+<around>3</around>
75+<regexp>^((ストップ高\t|ストップ安\t)?[,0-9]+|-{3})</regexp>
76+</searchlist>
77+<searchlist listNo="13">
78+<item>出来高</item>
79+<htmltag>div</htmltag>
80+<htmlclass>lineFi clearfix</htmlclass>
81+<around>4</around>
82+<regexp>^(.*?)\t</regexp>
83+</searchlist>
84+<searchlist listNo="14">
85+<item>売買代金</item>
86+<htmltag>div</htmltag>
87+<htmlclass>lineFi clearfix</htmlclass>
88+<around>5</around>
89+<regexp>^(.*?)\t</regexp>
90+</searchlist>
91+<searchlist listNo="15">
92+<item>値幅制限</item>
93+<htmltag>div</htmltag>
94+<htmlclass>lineFi clearfix</htmlclass>
95+<around>6</around>
96+<regexp>^(.*?)\t</regexp>
97+</searchlist>
98+<searchlist listNo="16">
99+<item>時価総額</item>
100+<htmltag>div</htmltag>
101+<htmlclass>lineFi yjMS clearfix</htmlclass>
102+<around>0</around>
103+<regexp>^(.*?)\t</regexp>
104+</searchlist>
105+<searchlist listNo="17">
106+<item>発行済株式数</item>
107+<htmltag>div</htmltag>
108+<htmlclass>lineFi yjMS clearfix</htmlclass>
109+<around>1</around>
110+<regexp>^(.*?)\t</regexp>
111+</searchlist>
112+<searchlist listNo="18">
113+<item>配当利回り</item>
114+<htmltag>div</htmltag>
115+<htmlclass>lineFi yjMS clearfix</htmlclass>
116+<around>2</around>
117+<regexp>^(.*?)\t</regexp>
118+</searchlist>
119+<searchlist listNo="19">
120+<item>１株配当</item>
121+<htmltag>div</htmltag>
122+<htmlclass>lineFi yjMS clearfix</htmlclass>
123+<around>3</around>
124+<regexp>^(.*?)\t</regexp>
125+</searchlist>
126+<searchlist listNo="20">
127+<item>PER</item>
128+<htmltag>div</htmltag>
129+<htmlclass>lineFi yjMS clearfix</htmlclass>
130+<around>4</around>
131+<regexp>^(.*?)\t</regexp>
132+</searchlist>
133+<searchlist listNo="21">
134+<item>PBR</item>
135+<htmltag>div</htmltag>
136+<htmlclass>lineFi yjMS clearfix</htmlclass>
137+<around>5</around>
138+<regexp>^(.*?)\t</regexp>
139+</searchlist>
140+<searchlist listNo="22">
141+<item>EPS</item>
142+<htmltag>div</htmltag>
143+<htmlclass>lineFi yjMS clearfix</htmlclass>
144+<around>6</around>
145+<regexp>^(.*?)\t</regexp>
146+</searchlist>
147+<searchlist listNo="23">
148+<item>BPS</item>
149+<htmltag>div</htmltag>
150+<htmlclass>lineFi yjMS clearfix</htmlclass>
151+<around>7</around>
152+<regexp>^(.*?)\t</regexp>
153+</searchlist>
154+<searchlist listNo="24">
155+<item>最低購入代金</item>
156+<htmltag>div</htmltag>
157+<htmlclass>lineFi yjMS clearfix</htmlclass>
158+<around>8</around>
159+<regexp>^(.*?)\t</regexp>
160+</searchlist>
161+<searchlist listNo="25">
162+<item>単元株数</item>
163+<htmltag>div</htmltag>
164+<htmlclass>lineFi yjMS clearfix</htmlclass>
165+<around>9</around>
166+<regexp>^(.*?)\t</regexp>
167+</searchlist>
168+<searchlist listNo="26">
169+<item>年初来高値</item>
170+<htmltag>div</htmltag>
171+<htmlclass>lineFi yjMS clearfix</htmlclass>
172+<around>10</around>
173+<regexp>^(.*?)\t</regexp>
174+</searchlist>
175+<searchlist listNo="27">
176+<item>年初来安値</item>
177+<htmltag>div</htmltag>
178+<htmlclass>lineFi yjMS clearfix</htmlclass>
179+<around>11</around>
180+<regexp>^(.*?)\t</regexp>
181+</searchlist>
182+<searchlist listNo="28">
183+<item>信用買残</item>
184+<htmltag>div</htmltag>
185+<htmlclass>lineFi yjMS clearfix</htmlclass>
186+<around>12</around>
187+<regexp>^(.*?)\t</regexp>
188+</searchlist>
189+<searchlist listNo="29">
190+<item>信用買残前週比</item>
191+<htmltag>div</htmltag>
192+<htmlclass>lineFi yjMS clearfix</htmlclass>
193+<around>13</around>
194+<regexp>^(.*?)\t</regexp>
195+</searchlist>
196+<searchlist listNo="30">
197+<item>信用売残</item>
198+<htmltag>div</htmltag>
199+<htmlclass>lineFi yjMS clearfix</htmlclass>
200+<around>14</around>
201+<regexp>^(.*?)\t</regexp>
202+</searchlist>
203+<searchlist listNo="31">
204+<item>信用売残前週比</item>
205+<htmltag>div</htmltag>
206+<htmlclass>lineFi yjMS clearfix</htmlclass>
207+<around>15</around>
208+<regexp>^(.*?)\t</regexp>
209+</searchlist>
210+<searchlist listNo="32">
211+<item>貸借倍率</item>
212+<htmltag>div</htmltag>
213+<htmlclass>yjMS clearfix</htmlclass>
214+<regexp>^(.*?)\t</regexp>
215+</searchlist>
216+</webscraping>
217+</xmlcontainer>
--- branches/b4/webScraping/data/Yahoo!天気.txt (nonexistent)
+++ branches/b4/webScraping/data/Yahoo!天気.txt (revision 134)
@@ -0,0 +1,14 @@
1+http://weather.yahoo.co.jp/weather/
2+天気０１ li point pt1400
3+天気０２ li point pt1900
4+天気０３ li point pt3410
5+天気０４ li point pt4410
6+天気０５ li point pt5110
7+天気０６ li point pt5410
8+天気０７ li point pt5610
9+天気０８ li point pt6200
10+天気０９ li point pt6710
11+天気１０ li point pt7410
12+天気１１ li point pt8210
13+天気１２ li point pt8810
14+天気１３ li point pt9110
--- branches/b4/webScraping/data/Yahoo!ファイナンス.txt (nonexistent)
+++ branches/b4/webScraping/data/Yahoo!ファイナンス.txt (revision 134)
@@ -0,0 +1,33 @@
1+http://stocks.finance.yahoo.co.jp/stocks/detail/?code=9984.T
2+銘柄コード dl stocksInfo clearFix (^\d{4})
3+カテゴリ div stockMainTabParts stockMainTabPartsCurrent
4+業種 dd category yjSb
5+取得時間 dd yjSb real ^(.*)\t
6+銘柄名 th symbol
7+株価 td stoksPrice
8+前日比 td change \t(.*)（.*%）
9+前日比％ td change \t.*（(.*)%）
10+前日終値 div lineFi clearfix 0 ^([,0-9]+)\t
11+始値 div lineFi clearfix 1 ^([,0-9]+|-{3})\t
12+高値 div lineFi clearfix 2 ^([,0-9]+|-{3})\t
13+安値 div lineFi clearfix 3 ^([,0-9]+|-{3})\t
14+出来高 div lineFi clearfix 4 ^(.*?)\t
15+売買代金 div lineFi clearfix 5 ^(.*?)\t
16+値幅制限 div lineFi clearfix 6 ^(.*?)\t
17+時価総額 div lineFi yjMS clearfix 0 ^(.*?)\t
18+発行済株式数 div lineFi yjMS clearfix 1 ^(.*?)\t
19+配当利回り div lineFi yjMS clearfix 2 ^(.*?)\t
20+１株配当 div lineFi yjMS clearfix 3 ^(.*?)\t
21+PER div lineFi yjMS clearfix 4 ^(.*?)\t
22+PBR div lineFi yjMS clearfix 5 ^(.*?)\t
23+EPS div lineFi yjMS clearfix 6 ^(.*?)\t
24+BPS div lineFi yjMS clearfix 7 ^(.*?)\t
25+最低購入代金 div lineFi yjMS clearfix 8 ^(.*?)\t
26+単元株数 div lineFi yjMS clearfix 9 ^(.*?)\t
27+年初来高値 div lineFi yjMS clearfix 10 ^(.*?)\t
28+年初来安値 div lineFi yjMS clearfix 11 ^(.*?)\t
29+信用買残 div lineFi yjMS clearfix 12 ^(.*?)\t
30+信用買残前週比 div lineFi yjMS clearfix 13 ^(.*?)\t
31+信用売残 div lineFi yjMS clearfix 14 ^(.*?)\t
32+信用売残前週比 div lineFi yjMS clearfix 15 ^(.*?)\t
33+貸借倍率 div yjMS clearfix ^(.*?)\t
--- branches/b4/webScraping/test/utility/test1/SearchDataRW.java (nonexistent)
+++ branches/b4/webScraping/test/utility/test1/SearchDataRW.java (revision 134)
@@ -0,0 +1,314 @@
1+/*
2+ * Copyright (C) 2014 kgto.
3+ *
4+ * This library is free software; you can redistribute it and/or
5+ * modify it under the terms of the GNU Lesser General Public
6+ * License as published by the Free Software Foundation; either
7+ * version 2.1 of the License, or (at your option) any later version.
8+ *
9+ * This library is distributed in the hope that it will be useful,
10+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
11+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
12+ * Lesser General Public License for more details.
13+ *
14+ * You should have received a copy of the GNU Lesser General Public
15+ * License along with this library; if not, write to the Free Software
16+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17+ * MA 02110-1301 USA
18+ */
19+/*
20+ * $Id$
21+ */
22+
23+package utility.test1;
24+
25+import webScraping.core.SearchData;
26+import java.io.File;
27+import java.io.FileNotFoundException;
28+import java.io.FileOutputStream;
29+import java.io.IOException;
30+import java.util.logging.Level;
31+import java.util.logging.Logger;
32+import javax.xml.parsers.DocumentBuilder;
33+import javax.xml.parsers.DocumentBuilderFactory;
34+import javax.xml.parsers.ParserConfigurationException;
35+import javax.xml.transform.Transformer;
36+import javax.xml.transform.TransformerConfigurationException;
37+import javax.xml.transform.TransformerException;
38+import javax.xml.transform.TransformerFactory;
39+import javax.xml.transform.dom.DOMSource;
40+import javax.xml.transform.stream.StreamResult;
41+import org.w3c.dom.DOMImplementation;
42+import org.w3c.dom.Document;
43+import org.w3c.dom.Element;
44+import org.w3c.dom.Node;
45+import org.w3c.dom.NodeList;
46+import org.xml.sax.SAXException;
47+
48+/**
49+ * 検索データ読込・保存.
50+ * @author kgto
51+ */
52+public class SearchDataRW {
53+ /* ---------------------------------------------------------------------- *
54+ * フィールド
55+ * ---------------------------------------------------------------------- */
56+ private String UrlAdress;
57+
58+ DocumentBuilder builder;
59+ public Document document;
60+ Element root;
61+
62+ /* ---------------------------------------------------------------------- *
63+ * コンストラクタ
64+ * ---------------------------------------------------------------------- */
65+ public SearchDataRW() {
66+ try {
67+ DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
68+ builder = factory.newDocumentBuilder();
69+
70+ } catch (ParserConfigurationException ex) {
71+ Logger.getLogger(SearchDataRW.class.getName()).log(Level.SEVERE, null, ex);
72+ }
73+ }
74+
75+ /* ---------------------------------------------------------------------- *
76+ * Setter
77+ * ---------------------------------------------------------------------- */
78+ public void seturl(String UrlAdress) {
79+ this.UrlAdress = UrlAdress;
80+ }
81+
82+ /* ---------------------------------------------------------------------- *
83+ * Getter
84+ * ---------------------------------------------------------------------- */
85+ public String geturl() {
86+ return UrlAdress;
87+ }
88+
89+ /* ---------------------------------------------------------------------- *
90+ * メソッド
91+ * ---------------------------------------------------------------------- */
92+ /**
93+ * 保存.
94+ * @param file
95+ */
96+ public void save(File file) {
97+ saveUrl(UrlAdress);
98+ saveSearchList();
99+ write(file);
100+ }
101+
102+ /**
103+ * 読込.
104+ * @param file
105+ */
106+ public void load(File file) {
107+ read(file);
108+ loadUrl();
109+ loadSearchList();
110+ }
111+
112+ /* ---------------------------------------------------------------------- */
113+
114+ void loadUrl() {
115+ NodeList nodelist = root.getElementsByTagName("url");
116+ Node node = nodelist.item(0);
117+ UrlAdress = node.getFirstChild().getNodeValue();
118+ }
119+
120+ public void loadSearchList() {
121+ SearchData.clear();
122+
123+ NodeList nodelist = root.getElementsByTagName("searchlist");
124+ for(int i = 0; i < nodelist.getLength(); i++) {
125+ Node childnode = nodelist.item(i);
126+
127+ boolean sdatflg = false;
128+ SearchData sdat = new SearchData();
129+ for (Node child = childnode.getFirstChild(); child != null; child = child.getNextSibling()) {
130+ if(child.getNodeType() == Node.ELEMENT_NODE) {
131+ String tag = child.getNodeName();
132+ String rtn = "";
133+ if(child.getFirstChild() != null) {
134+ rtn = child.getFirstChild().getNodeValue();
135+ }
136+ switch (tag) {
137+ case "item" :
138+ sdat.setitem(rtn);
139+ sdatflg = true;
140+ break;
141+ case "htmltag" :
142+ sdat.setHtmltag(rtn);
143+ sdatflg = true;
144+ break;
145+ case "htmlid" :
146+ sdat.setHtmlid(rtn);
147+ sdatflg = true;
148+ break;
149+ case "htmlclass" :
150+ sdat.setHtmlclass(rtn);
151+ sdatflg = true;
152+ break;
153+ case "around" :
154+ sdat.setaround(rtn);
155+ sdatflg = true;
156+ break;
157+ case "regexp" :
158+ sdat.setregexp(rtn);
159+ sdatflg = true;
160+ break;
161+ }
162+ }
163+ }
164+ if(sdatflg) SearchData.add(sdat);
165+ }
166+ }
167+
168+ public String loadMsg404() {
169+ StringBuilder strbuf = new StringBuilder();
170+ NodeList nodelist = root.getElementsByTagName("msg404");
171+ for(int i = 0; i < nodelist.getLength(); i++) {
172+ Node childnode = nodelist.item(i);
173+ String str = childnode.getFirstChild().getNodeValue();
174+ if(strbuf.length() > 0) {
175+ strbuf.append("\n");
176+ }
177+ strbuf.append(str);
178+ }
179+ return strbuf.toString();
180+ }
181+
182+ public Element loadElement(String elementTagName) {
183+ NodeList nodelist = root.getElementsByTagName(elementTagName);
184+ Element element = (Element)nodelist.item(0);
185+
186+ return element;
187+ }
188+
189+ /* ---------------------------------------------------------------------- */
190+
191+ void saveUrl(String urladdress) {
192+ checkdoc();
193+ removeElement("url"); // 既にElementが存在してた場合、一度削除
194+
195+ Element url = document.createElement("url");
196+ url.appendChild(document.createTextNode(urladdress));
197+ root.appendChild(url);
198+ }
199+
200+ void saveSearchList() {
201+ checkdoc();
202+ removeElement("searchlist"); // 既にElementが存在してた場合、一度削除
203+
204+ int count = 0;
205+ for(int i = 0; i < SearchData.size(); i++) {
206+ SearchData sdat = SearchData.get(i);
207+
208+ Element cslist = document.createElement("searchlist");
209+ cslist.setAttribute("listNo", String.valueOf(++count));
210+
211+ addChild(cslist, "item", sdat.getitem());
212+ addChild(cslist, "htmltag", sdat.getHtmltag());
213+ addChild(cslist, "htmlid", sdat.getHtmlid());
214+ addChild(cslist, "htmlclass", sdat.getHtmlclass());
215+ addChild(cslist, "around", sdat.getaround());
216+ addChild(cslist, "regexp", sdat.getregexp());
217+
218+ root.appendChild(cslist);
219+ }
220+ }
221+
222+ void saveMsg404(String msg) {
223+ checkdoc();
224+ removeElement("msg404"); // 既にElementが存在してた場合、一度削除
225+
226+ String[] msgs = msg.split("\n");
227+ int count = 0;
228+ for(String msgOne : msgs) {
229+ Element msgElement = document.createElement("msg404");
230+ msgElement.setAttribute("No", String.valueOf(++count));
231+ msgElement.appendChild(document.createTextNode(msgOne));
232+
233+ root.appendChild(msgElement);
234+ }
235+ }
236+
237+ public void saveElement(Element element) {
238+ checkdoc();
239+ removeElement(element.getTagName()); // 既にElementが存在してた場合、一度削除
240+
241+ root.appendChild(element);
242+ }
243+
244+ /* ---------------------------------------------------------------------- */
245+
246+ private void addChild(Element cslist, String keyword, String data) {
247+ if(!data.isEmpty()) {
248+ Element element = document.createElement(keyword);
249+ element.appendChild(document.createTextNode(data));
250+ cslist.appendChild(element);
251+ }
252+ }
253+
254+ private void removeElement(String elementTagName) {
255+ int nodeSize;
256+ do {
257+ NodeList nodelist = document.getElementsByTagName(elementTagName);
258+ nodeSize = nodelist.getLength();
259+ for(int i = 0; i < nodelist.getLength(); i++) {
260+ Node node = nodelist.item(i);
261+ root.removeChild(node);
262+ }
263+ } while(nodeSize > 0);
264+ }
265+
266+ /**
267+ * ドキュメントチェック.
268+ * 新規の場合やＸＭＬファイルの読込みが行われていない状態時、新たにルートエレメントを作成する。
269+ * 既読の場合、ルートエレメントの取得を行う。
270+ */
271+ public void checkdoc() {
272+ if(document == null) {
273+ DOMImplementation domImpl = builder.getDOMImplementation();
274+ document = domImpl.createDocument("","searchdata",null);
275+ }
276+ root = document.getDocumentElement();
277+ }
278+
279+ /**
280+ * XML読込み.
281+ * @param file
282+ */
283+ public void read(File file) {
284+ try {
285+ document = builder.parse(file);
286+ root = document.getDocumentElement();
287+
288+ } catch (SAXException | IOException ex) {
289+ Logger.getLogger(SearchDataRW.class.getName()).log(Level.SEVERE, null, ex);
290+ }
291+ }
292+
293+ /**
294+ * XML書込み.
295+ * @param file
296+ */
297+ public void write(File file) {
298+ try {
299+ TransformerFactory transFactory = TransformerFactory.newInstance();
300+ Transformer transformer = transFactory.newTransformer();
301+
302+ DOMSource source = new DOMSource(document);
303+ FileOutputStream os = new FileOutputStream(file);
304+ StreamResult result = new StreamResult(os);
305+ transformer.transform(source, result);
306+
307+ } catch (TransformerConfigurationException ex) {
308+ Logger.getLogger(SearchDataRW.class.getName()).log(Level.SEVERE, null, ex);
309+ } catch (FileNotFoundException | TransformerException ex) {
310+ Logger.getLogger(SearchDataRW.class.getName()).log(Level.SEVERE, null, ex);
311+ }
312+ }
313+
314+}
Added: svn:keywords
## -0,0 +1 ##
+Id
\ No newline at end of property
--- branches/b4/webScraping/test/utility/test1/SearchDataRWT01.java (nonexistent)
+++ branches/b4/webScraping/test/utility/test1/SearchDataRWT01.java (revision 134)
@@ -0,0 +1,65 @@
1+
2+package utility.test1;
3+
4+import java.io.File;
5+import java.lang.reflect.InvocationTargetException;
6+import java.lang.reflect.Method;
7+import java.util.logging.Level;
8+import java.util.logging.Logger;
9+
10+/**
11+ * ファイル書出し・読込みテスト
12+ * @author kgto
13+ */
14+public class SearchDataRWT01 {
15+ SearchDataRW sio = new SearchDataRW();
16+
17+ File file = new File("SearchDataRWT01.xml");
18+
19+ /**
20+ * @param args the command line arguments
21+ */
22+ public static void main(String[] args) {
23+ SearchDataRWT01 test01 = new SearchDataRWT01();
24+ test01.save01();
25+ test01.load01();
26+ }
27+
28+ void SearchDataRWT01() {
29+ }
30+
31+ void save01() {
32+ try {
33+ String str = "abc\ndef\nghi\n1111";
34+
35+ //sio.saveMsg404(str);
36+ // リフレクション
37+ Method method = SearchDataRW.class.getDeclaredMethod("saveMsg404", String.class);
38+ method.setAccessible(true);
39+ method.invoke(sio, str);
40+
41+ sio.write(file);
42+
43+ } catch (NoSuchMethodException | SecurityException
44+ | IllegalAccessException | IllegalArgumentException | InvocationTargetException ex) {
45+ Logger.getLogger(SearchDataRWT01.class.getName()).log(Level.SEVERE, null, ex);
46+ }
47+ }
48+
49+ void load01() {
50+ try {
51+ Method method = SearchDataRW.class.getDeclaredMethod("loadMsg404");
52+ method.setAccessible(true);
53+ Object obj = method.invoke(sio);
54+ String str = (String)obj;
55+
56+ System.out.println("loadMsg404 = " + str);
57+
58+
59+ } catch (NoSuchMethodException | SecurityException
60+ | IllegalAccessException | IllegalArgumentException | InvocationTargetException ex) {
61+ Logger.getLogger(SearchDataRWT01.class.getName()).log(Level.SEVERE, null, ex);
62+ }
63+ }
64+
65+}
--- branches/b4/webScraping/test/utility/test1/SearchDataRWT02.java (nonexistent)
+++ branches/b4/webScraping/test/utility/test1/SearchDataRWT02.java (revision 134)
@@ -0,0 +1,45 @@
1+
2+package utility.test1;
3+
4+import java.io.File;
5+import webScraping.core.SearchData;
6+
7+/**
8+ * ファイル読込みテスト
9+ * @author kgto
10+ */
11+public class SearchDataRWT02 {
12+ SearchDataRW sio = new SearchDataRW();
13+
14+ File file = new File("test1.xml");
15+
16+ /**
17+ * @param args the command line arguments
18+ */
19+ public static void main(String[] args) {
20+ SearchDataRWT02 test01 = new SearchDataRWT02();
21+ test01.load01();
22+ }
23+
24+ void SearchDataRWT01() {
25+ }
26+
27+ void load01() {
28+ char spchar = '\t';
29+
30+ sio.load(file);
31+ for(int i = 0; i < SearchData.size(); i++) {
32+ SearchData sdat = SearchData.get(i);
33+
34+ StringBuilder sbuf = new StringBuilder();
35+ sbuf.append(sdat.getitem()).append(spchar);
36+ sbuf.append(sdat.getHtmltag()).append(spchar);
37+ sbuf.append(sdat.getHtmlid()).append(spchar);
38+ sbuf.append(sdat.getHtmlclass()).append(spchar);
39+ sbuf.append(sdat.getaround()).append(spchar);
40+ sbuf.append(sdat.getregexp()).append(spchar);
41+ System.out.println(sbuf.toString());
42+ }
43+ }
44+
45+}
--- branches/b4/webScraping/test/utility/test1/ConvertXml01.java (nonexistent)
+++ branches/b4/webScraping/test/utility/test1/ConvertXml01.java (revision 134)
@@ -0,0 +1,42 @@
1+
2+package utility.test1;
3+
4+import java.io.File;
5+import webScraping.utility.ScrapingXml;
6+
7+/**
8+ * XMLコンバータ
9+ * 旧:SearchDataRW.java → 新:ScrapingXml.java
10+ * @author kgto
11+ */
12+public class ConvertXml01 {
13+
14+ private String UrlAdress;
15+ File file = new File("test1.xml");
16+
17+ /**
18+ * @param args the command line arguments
19+ */
20+ public static void main(String[] args) {
21+ ConvertXml01 conv = new ConvertXml01();
22+
23+ conv.readold();
24+ conv.writenew();
25+
26+ System.exit(0);
27+ }
28+
29+ void readold() {
30+ SearchDataRW sdatrw = new SearchDataRW();
31+ sdatrw.load(file);
32+ UrlAdress = sdatrw.geturl();
33+ }
34+
35+ void writenew() {
36+ ScrapingXml xmlwriter = new ScrapingXml();
37+ xmlwriter.setTestUrl(UrlAdress);
38+ xmlwriter.setSdata();
39+ xmlwriter.save(file);
40+ }
41+
42+}
--- branches/b4/webScraping/test1.xml (nonexistent)
+++ branches/b4/webScraping/test1.xml (revision 134)
@@ -0,0 +1,71 @@
1+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
2+<xmlcontainer>
3+<webscraping>
4+<url>http://weather.yahoo.co.jp/weather/</url>
5+<searchlist listNo="1">
6+<item>天気０１</item>
7+<htmltag>li</htmltag>
8+<htmlclass>point pt1400</htmlclass>
9+</searchlist>
10+<searchlist listNo="2">
11+<item>天気０２</item>
12+<htmltag>li</htmltag>
13+<htmlclass>point pt1900</htmlclass>
14+</searchlist>
15+<searchlist listNo="3">
16+<item>天気０３</item>
17+<htmltag>li</htmltag>
18+<htmlclass>point pt3410</htmlclass>
19+</searchlist>
20+<searchlist listNo="4">
21+<item>天気０４</item>
22+<htmltag>li</htmltag>
23+<htmlclass>point pt4410</htmlclass>
24+</searchlist>
25+<searchlist listNo="5">
26+<item>天気０５</item>
27+<htmltag>li</htmltag>
28+<htmlclass>point pt5110</htmlclass>
29+</searchlist>
30+<searchlist listNo="6">
31+<item>天気０６</item>
32+<htmltag>li</htmltag>
33+<htmlclass>point pt5410</htmlclass>
34+</searchlist>
35+<searchlist listNo="7">
36+<item>天気０７</item>
37+<htmltag>li</htmltag>
38+<htmlclass>point pt5610</htmlclass>
39+</searchlist>
40+<searchlist listNo="8">
41+<item>天気０８</item>
42+<htmltag>li</htmltag>
43+<htmlclass>point pt6200</htmlclass>
44+</searchlist>
45+<searchlist listNo="9">
46+<item>天気０９</item>
47+<htmltag>li</htmltag>
48+<htmlclass>point pt6710</htmlclass>
49+</searchlist>
50+<searchlist listNo="10">
51+<item>天気１０</item>
52+<htmltag>li</htmltag>
53+<htmlclass>point pt7410</htmlclass>
54+</searchlist>
55+<searchlist listNo="11">
56+<item>天気１１</item>
57+<htmltag>li</htmltag>
58+<htmlclass>point pt8210</htmlclass>
59+</searchlist>
60+<searchlist listNo="12">
61+<item>天気１２</item>
62+<htmltag>li</htmltag>
63+<htmlclass>point pt8810</htmlclass>
64+</searchlist>
65+<searchlist listNo="13">
66+<item>天気１３</item>
67+<htmltag>li</htmltag>
68+<htmlclass>point pt9110</htmlclass>
69+</searchlist>
70+</webscraping>
71+</xmlcontainer>
--- branches/b4/webScraping/src/WebScraping/utility/LibraryXml.java (nonexistent)
+++ branches/b4/webScraping/src/WebScraping/utility/LibraryXml.java (revision 134)
@@ -0,0 +1,142 @@
1+/*
2+ * Copyright (C) 2014-2015 kgto.
3+ *
4+ * This library is free software; you can redistribute it and/or
5+ * modify it under the terms of the GNU Lesser General Public
6+ * License as published by the Free Software Foundation; either
7+ * version 2.1 of the License, or (at your option) any later version.
8+ *
9+ * This library is distributed in the hope that it will be useful,
10+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
11+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
12+ * Lesser General Public License for more details.
13+ *
14+ * You should have received a copy of the GNU Lesser General Public
15+ * License along with this library; if not, write to the Free Software
16+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17+ * MA 02110-1301 USA
18+ */
19+/*
20+ * $Id$
21+ */
22+
23+package webScraping.utility;
24+
25+import java.io.File;
26+import java.io.FileNotFoundException;
27+import java.io.FileOutputStream;
28+import java.io.IOException;
29+import java.util.logging.Level;
30+import java.util.logging.Logger;
31+
32+import javax.xml.parsers.DocumentBuilder;
33+import javax.xml.parsers.DocumentBuilderFactory;
34+import javax.xml.parsers.ParserConfigurationException;
35+import javax.xml.transform.Transformer;
36+import javax.xml.transform.TransformerConfigurationException;
37+import javax.xml.transform.TransformerException;
38+import javax.xml.transform.TransformerFactory;
39+import javax.xml.transform.dom.DOMSource;
40+import javax.xml.transform.stream.StreamResult;
41+
42+import org.w3c.dom.DOMImplementation;
43+import org.w3c.dom.Document;
44+import org.w3c.dom.Element;
45+import org.w3c.dom.Node;
46+import org.w3c.dom.NodeList;
47+import org.xml.sax.SAXException;
48+
49+public class LibraryXml {
50+
51+ String xmlrootname = "xmlcontainer";
52+
53+ DocumentBuilder builder;
54+ public Document readdoc, writedoc;
55+ Element xmlroot;
56+
57+ /* ---------------------------------------------------------------------- *
58+ * コンストラクタ
59+ * ---------------------------------------------------------------------- */
60+ public LibraryXml() {
61+ try {
62+ DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
63+ builder = factory.newDocumentBuilder();
64+
65+ } catch (ParserConfigurationException ex) {
66+ Logger.getLogger(LibraryXml.class.getName()).log(Level.SEVERE, null, ex);
67+ }
68+ }
69+
70+ /* ---------------------------------------------------------------------- *
71+ * メソッド
72+ * ---------------------------------------------------------------------- */
73+ /* 読込み処理 */
74+ public Element getwriteRoot(String elementName) {
75+ mainElement();
76+ Element element = writedoc.createElement(elementName);
77+ xmlroot.appendChild(element);
78+ return element;
79+ }
80+
81+ private void mainElement() {
82+ if(writedoc == null) {
83+ DOMImplementation domImpl = builder.getDOMImplementation();
84+ writedoc = domImpl.createDocument("", xmlrootname, null);
85+ xmlroot = writedoc.getDocumentElement();
86+ }
87+ }
88+
89+ /**
90+ * XML書込み.
91+ * @param file
92+ */
93+ public void write(File file) {
94+ try (FileOutputStream os = new FileOutputStream(file)) {
95+ TransformerFactory transFactory = TransformerFactory.newInstance();
96+ Transformer transformer = transFactory.newTransformer();
97+
98+ transformer.setOutputProperty("indent", "yes"); // 改行指定
99+ transformer.setOutputProperty("method", "xml");
100+
101+ DOMSource source = new DOMSource(writedoc);
102+ StreamResult result = new StreamResult(os);
103+ transformer.transform(source, result);
104+
105+ // 作成したＸＭＬをクリア
106+ writedoc = null;
107+
108+ } catch (TransformerConfigurationException ex) {
109+ Logger.getLogger(LibraryXml.class.getName()).log(Level.SEVERE, null, ex);
110+ } catch (FileNotFoundException | TransformerException ex) {
111+ Logger.getLogger(LibraryXml.class.getName()).log(Level.SEVERE, null, ex);
112+ } catch (IOException ex) {
113+ Logger.getLogger(LibraryXml.class.getName()).log(Level.SEVERE, null, ex);
114+ }
115+ }
116+
117+ /* ---------------------------------------------------------------------- */
118+ /* 書込み処理 */
119+
120+ public Element getreadRoot(String elementName) {
121+ NodeList nodelist = xmlroot.getElementsByTagName(elementName);
122+ Node node = nodelist.item(0);
123+ return (node.getNodeType() == Node.ELEMENT_NODE ? (Element)node : null);
124+ }
125+
126+ /**
127+ * XML読込み.
128+ * @param file
129+ */
130+ public void read(File file) {
131+ try {
132+ readdoc = builder.parse(file);
133+ xmlroot = readdoc.getDocumentElement();
134+
135+ } catch (SAXException | IOException ex) {
136+ Logger.getLogger(LibraryXml.class.getName()).log(Level.SEVERE, null, ex);
137+ }
138+ }
139+
140+ /* ---------------------------------------------------------------------- */
141+
142+}
Added: svn:keywords
## -0,0 +1 ##
+Id
\ No newline at end of property
--- branches/b4/webScraping/src/WebScraping/utility/ScrapingXml.java (nonexistent)
+++ branches/b4/webScraping/src/WebScraping/utility/ScrapingXml.java (revision 134)
@@ -0,0 +1,198 @@
1+/*
2+ * Copyright (C) 2014-2015 kgto.
3+ *
4+ * This library is free software; you can redistribute it and/or
5+ * modify it under the terms of the GNU Lesser General Public
6+ * License as published by the Free Software Foundation; either
7+ * version 2.1 of the License, or (at your option) any later version.
8+ *
9+ * This library is distributed in the hope that it will be useful,
10+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
11+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
12+ * Lesser General Public License for more details.
13+ *
14+ * You should have received a copy of the GNU Lesser General Public
15+ * License along with this library; if not, write to the Free Software
16+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17+ * MA 02110-1301 USA
18+ */
19+/*
20+ * $Id$
21+ */
22+
23+package webScraping.utility;
24+
25+import webScraping.core.SearchData;
26+import java.io.File;
27+import java.util.ArrayList;
28+import org.w3c.dom.Element;
29+import org.w3c.dom.Node;
30+import org.w3c.dom.NodeList;
31+
32+public class ScrapingXml {
33+ /* ---------------------------------------------------------------------- *
34+ * フィールド
35+ * ---------------------------------------------------------------------- */
36+ String rootnameScraping = "webscraping";
37+
38+ private String testUrl;
39+ private SearchData[] sdata;
40+
41+ public LibraryXml xlib = new LibraryXml();
42+ public Element root;
43+
44+ /* ---------------------------------------------------------------------- *
45+ * コンストラクタ
46+ * ---------------------------------------------------------------------- */
47+ public ScrapingXml() {
48+ }
49+
50+ /* ---------------------------------------------------------------------- *
51+ * Setter
52+ * ---------------------------------------------------------------------- */
53+ public void setTestUrl(String testUrl) {
54+ this.testUrl = testUrl;
55+ }
56+
57+ public void setSdata() {
58+ this.sdata = new SearchData[SearchData.size()];
59+ for(int i = 0; i < SearchData.size(); i++) {
60+ this.sdata[i] = SearchData.get(i);
61+ }
62+ }
63+
64+ /* ---------------------------------------------------------------------- *
65+ * Getter
66+ * ---------------------------------------------------------------------- */
67+ public String getTestUrl() {
68+ return testUrl;
69+ }
70+
71+ public void getSdata() {
72+ SearchData.clear();
73+ for(SearchData sdata1 : sdata) {
74+ SearchData.add(sdata1);
75+ }
76+ }
77+
78+ /* ---------------------------------------------------------------------- *
79+ * メソッド
80+ * ---------------------------------------------------------------------- */
81+ public void save(File file) {
82+
83+ elementset();
84+
85+ xlib.write(file);
86+ }
87+
88+ public void elementset() {
89+ root = xlib.getwriteRoot(rootnameScraping);
90+ elementsetUrl();
91+ elementsetSearchdata();
92+ System.out.println("elementset XmlScraping");
93+ }
94+
95+ private void elementsetUrl() {
96+ Element url = xlib.writedoc.createElement("url");
97+ url.appendChild(xlib.writedoc.createTextNode(testUrl));
98+ root.appendChild(url);
99+ }
100+
101+ private void elementsetSearchdata() {
102+ int count = 0;
103+ for(SearchData sdat : sdata) {
104+ Element cslist = xlib.writedoc.createElement("searchlist");
105+ cslist.setAttribute("listNo", String.valueOf(++count));
106+
107+ addChild(cslist, "item" , sdat.getitem());
108+ addChild(cslist, "htmltag" , sdat.getHtmltag());
109+ addChild(cslist, "htmlid" , sdat.getHtmlid());
110+ addChild(cslist, "htmlclass", sdat.getHtmlclass());
111+ addChild(cslist, "around" , sdat.getaround());
112+ addChild(cslist, "regexp" , sdat.getregexp());
113+
114+ root.appendChild(cslist);
115+ }
116+ }
117+
118+ private void addChild(Element cslist, String keyword, String data) {
119+ if(!data.isEmpty()) {
120+ Element element = xlib.writedoc.createElement(keyword);
121+ element.appendChild(xlib.writedoc.createTextNode(data));
122+ cslist.appendChild(element);
123+ }
124+ }
125+
126+ /* ---------------------------------------------------------------------- */
127+
128+ void load(File file) {
129+ xlib.read(file);
130+ elementget();
131+ }
132+
133+ public void elementget() {
134+ root = xlib.getreadRoot(rootnameScraping);
135+ elementgetUrl();
136+ elementgetSearchdata();
137+ }
138+
139+ private void elementgetUrl() {
140+ NodeList nodelist = root.getElementsByTagName("url");
141+ Node node = nodelist.item(0);
142+ testUrl = node.getFirstChild().getNodeValue();
143+ }
144+
145+ private void elementgetSearchdata() {
146+ ArrayList<SearchData> slist = new ArrayList<>();
147+
148+ NodeList nodelist = root.getElementsByTagName("searchlist");
149+ for(int i = 0; i < nodelist.getLength(); i++) {
150+ Node childnode = nodelist.item(i);
151+
152+ boolean sdatflg = false;
153+ SearchData sdat = new SearchData();
154+ for (Node child = childnode.getFirstChild(); child != null; child = child.getNextSibling()) {
155+ if(child.getNodeType() == Node.ELEMENT_NODE) {
156+ String tag = child.getNodeName();
157+ String rtn = "";
158+ if(child.getFirstChild() != null) {
159+ rtn = child.getFirstChild().getNodeValue();
160+ }
161+ switch (tag) {
162+ case "item" :
163+ sdat.setitem(rtn);
164+ sdatflg = true;
165+ break;
166+ case "htmltag" :
167+ sdat.setHtmltag(rtn);
168+ sdatflg = true;
169+ break;
170+ case "htmlid" :
171+ sdat.setHtmlid(rtn);
172+ sdatflg = true;
173+ break;
174+ case "htmlclass" :
175+ sdat.setHtmlclass(rtn);
176+ sdatflg = true;
177+ break;
178+ case "around" :
179+ sdat.setaround(rtn);
180+ sdatflg = true;
181+ break;
182+ case "regexp" :
183+ sdat.setregexp(rtn);
184+ sdatflg = true;
185+ break;
186+ }
187+ }
188+ }
189+ if(sdatflg) slist.add(sdat);
190+ }
191+ // 配列化
192+ sdata = new SearchData[slist.size()];
193+ for(int i = 0; i < slist.size(); i++) {
194+ sdata[i] = slist.get(i);
195+ }
196+ }
197+
198+}
Added: svn:keywords
## -0,0 +1 ##
+Id
\ No newline at end of property
--- branches/b4/webScraping/src/WebScraping/utility/HtmlSearch.java (nonexistent)
+++ branches/b4/webScraping/src/WebScraping/utility/HtmlSearch.java (revision 134)
@@ -0,0 +1,568 @@
1+/*
2+ * Copyright (C) 2014 kgto.
3+ *
4+ * This library is free software; you can redistribute it and/or
5+ * modify it under the terms of the GNU Lesser General Public
6+ * License as published by the Free Software Foundation; either
7+ * version 2.1 of the License, or (at your option) any later version.
8+ *
9+ * This library is distributed in the hope that it will be useful,
10+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
11+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
12+ * Lesser General Public License for more details.
13+ *
14+ * You should have received a copy of the GNU Lesser General Public
15+ * License along with this library; if not, write to the Free Software
16+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17+ * MA 02110-1301 USA
18+ */
19+/*
20+ * $Id$
21+ */
22+package webScraping.utility;
23+
24+import webScraping.core.HtmlParser;
25+import webScraping.core.SearchData;
26+import java.awt.Desktop;
27+import java.io.File;
28+import java.io.IOException;
29+import java.net.URI;
30+import java.net.URISyntaxException;
31+import java.util.logging.Level;
32+import java.util.logging.Logger;
33+import javax.swing.JFileChooser;
34+import javax.swing.filechooser.FileFilter;
35+import javax.swing.filechooser.FileNameExtensionFilter;
36+import javax.swing.table.DefaultTableModel;
37+
38+/**
39+ * HTMLページ上の特定の項目を検索し、その項目内容の値を取得する.
40+ * @author kgto
41+ */
42+public class HtmlSearch extends javax.swing.JFrame {
43+ private final ScrapingXml xmlwriter = new ScrapingXml();
44+
45+ SearchDataTableModel sdatatblmodel;
46+
47+ /**
48+ * Creates new form Frame1
49+ */
50+ public HtmlSearch() {
51+ sdatatblmodel = new SearchDataTableModel();
52+
53+ initComponents();
54+
55+ // カレントディレクトリ取得
56+ String dir = System.getProperty("user.dir");
57+ File file = new java.io.File(dir + "\\data");
58+ jFileChooser1.setCurrentDirectory(file);
59+
60+ FileFilter filter1 = new FileNameExtensionFilter("XMLファイル", "xml");
61+ FileFilter filter2 = new FileNameExtensionFilter("TEXTファイル", "txt");
62+ jFileChooser1.addChoosableFileFilter(filter1);
63+ jFileChooser1.addChoosableFileFilter(filter2);
64+ jFileChooser1.setFileFilter(filter1);
65+
66+ }
67+
68+ /**
69+ * This method is called from within the constructor to initialize the form.
70+ * WARNING: Do NOT modify this code. The content of this method is always
71+ * regenerated by the Form Editor.
72+ */
73+ @SuppressWarnings("unchecked")
74+ // <editor-fold defaultstate="collapsed" desc="Generated Code">//GEN-BEGIN:initComponents
75+ private void initComponents() {
76+
77+ jFileChooser1 = new javax.swing.JFileChooser();
78+ jRadioButton1 = new javax.swing.JRadioButton();
79+ jLabel1 = new javax.swing.JLabel();
80+ jTxtUrl = new javax.swing.JTextField();
81+ jBtnSearch = new javax.swing.JButton();
82+ jTabbedPane1 = new javax.swing.JTabbedPane();
83+ jPanelTab1 = new javax.swing.JPanel();
84+ jScrollPane1 = new javax.swing.JScrollPane();
85+ jTable1 = new javax.swing.JTable();
86+ jBtnRowIns = new javax.swing.JButton();
87+ jBtnRowDel = new javax.swing.JButton();
88+ jBtnRowCpy = new javax.swing.JButton();
89+ jPanelTab2 = new javax.swing.JPanel();
90+ jScrollPaneLabel = new javax.swing.JScrollPane();
91+ jTxtLabel = new javax.swing.JTextArea();
92+ jScrollPane404msg = new javax.swing.JScrollPane();
93+ jTxt404msg = new javax.swing.JTextArea();
94+ jPanelRtn = new javax.swing.JPanel();
95+ jScrollPaneRtn = new javax.swing.JScrollPane();
96+ jTxtRtn = new javax.swing.JTextArea();
97+ jMenuBar1 = new javax.swing.JMenuBar();
98+ jMenu1 = new javax.swing.JMenu();
99+ jMenuLoad = new javax.swing.JMenuItem();
100+ jMenuSave = new javax.swing.JMenuItem();
101+ jMenu3 = new javax.swing.JMenu();
102+ jMenuItem1 = new javax.swing.JMenuItem();
103+ jMenu2 = new javax.swing.JMenu();
104+
105+ jFileChooser1.setCurrentDirectory(null);
106+ jFileChooser1.setDialogTitle("");
107+
108+ jRadioButton1.setText("jRadioButton1");
109+
110+ setDefaultCloseOperation(javax.swing.WindowConstants.EXIT_ON_CLOSE);
111+ setTitle("タグ検索");
112+
113+ jLabel1.setText(" URL:");
114+
115+ jBtnSearch.setText("検索");
116+ jBtnSearch.addActionListener(new java.awt.event.ActionListener() {
117+ public void actionPerformed(java.awt.event.ActionEvent evt) {
118+ jBtnSearchActionPerformed(evt);
119+ }
120+ });
121+
122+ jPanelTab1.setBorder(javax.swing.BorderFactory.createTitledBorder("検索情報"));
123+
124+ jTable1.setModel(sdatatblmodel);
125+ jTable1.setSelectionMode(javax.swing.ListSelectionModel.SINGLE_SELECTION);
126+ jTable1.getTableHeader().setReorderingAllowed(false);
127+ jScrollPane1.setViewportView(jTable1);
128+
129+ jBtnRowIns.setText("行挿入");
130+ jBtnRowIns.addActionListener(new java.awt.event.ActionListener() {
131+ public void actionPerformed(java.awt.event.ActionEvent evt) {
132+ jBtnRowInsActionPerformed(evt);
133+ }
134+ });
135+
136+ jBtnRowDel.setText("行削除");
137+ jBtnRowDel.addActionListener(new java.awt.event.ActionListener() {
138+ public void actionPerformed(java.awt.event.ActionEvent evt) {
139+ jBtnRowDelActionPerformed(evt);
140+ }
141+ });
142+
143+ jBtnRowCpy.setText("行コピー");
144+ jBtnRowCpy.addActionListener(new java.awt.event.ActionListener() {
145+ public void actionPerformed(java.awt.event.ActionEvent evt) {
146+ jBtnRowCpyActionPerformed(evt);
147+ }
148+ });
149+
150+ javax.swing.GroupLayout jPanelTab1Layout = new javax.swing.GroupLayout(jPanelTab1);
151+ jPanelTab1.setLayout(jPanelTab1Layout);
152+ jPanelTab1Layout.setHorizontalGroup(
153+ jPanelTab1Layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
154+ .addGroup(jPanelTab1Layout.createSequentialGroup()
155+ .addContainerGap(javax.swing.GroupLayout.DEFAULT_SIZE, Short.MAX_VALUE)
156+ .addComponent(jBtnRowCpy)
157+ .addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.RELATED)
158+ .addComponent(jBtnRowDel)
159+ .addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.RELATED)
160+ .addComponent(jBtnRowIns))
161+ .addComponent(jScrollPane1, javax.swing.GroupLayout.PREFERRED_SIZE, 0, Short.MAX_VALUE)
162+ );
163+ jPanelTab1Layout.setVerticalGroup(
164+ jPanelTab1Layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
165+ .addGroup(jPanelTab1Layout.createSequentialGroup()
166+ .addComponent(jScrollPane1, javax.swing.GroupLayout.DEFAULT_SIZE, 173, Short.MAX_VALUE)
167+ .addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.RELATED)
168+ .addGroup(jPanelTab1Layout.createParallelGroup(javax.swing.GroupLayout.Alignment.BASELINE)
169+ .addComponent(jBtnRowDel)
170+ .addComponent(jBtnRowIns)
171+ .addComponent(jBtnRowCpy)))
172+ );
173+
174+ jTabbedPane1.addTab("キー設定", jPanelTab1);
175+
176+ jPanelTab2.setBorder(javax.swing.BorderFactory.createTitledBorder("メッセージ"));
177+
178+ jScrollPaneLabel.setHorizontalScrollBarPolicy(javax.swing.ScrollPaneConstants.HORIZONTAL_SCROLLBAR_NEVER);
179+ jScrollPaneLabel.setVerticalScrollBarPolicy(javax.swing.ScrollPaneConstants.VERTICAL_SCROLLBAR_NEVER);
180+
181+ jTxtLabel.setEditable(false);
182+ jTxtLabel.setBackground(java.awt.Color.lightGray);
183+ jTxtLabel.setColumns(20);
184+ jTxtLabel.setFont(new java.awt.Font("MS UI Gothic", 0, 12)); // NOI18N
185+ jTxtLabel.setLineWrap(true);
186+ jTxtLabel.setRows(2);
187+ jTxtLabel.setText("取得ページに以下のメッセージが含まれていた場合、対象データが取得出来なかったと通知します。");
188+ jTxtLabel.setAutoscrolls(false);
189+ jTxtLabel.setBorder(null);
190+ jTxtLabel.setCursor(new java.awt.Cursor(java.awt.Cursor.DEFAULT_CURSOR));
191+ jTxtLabel.setFocusable(false);
192+ jTxtLabel.setHighlighter(null);
193+ jTxtLabel.setKeymap(null);
194+ jTxtLabel.setOpaque(false);
195+ jTxtLabel.setRequestFocusEnabled(false);
196+ jTxtLabel.setVerifyInputWhenFocusTarget(false);
197+ jScrollPaneLabel.setViewportView(jTxtLabel);
198+
199+ jTxt404msg.setColumns(20);
200+ jTxt404msg.setRows(3);
201+ jTxt404msg.setText("一致する銘柄は見つかりませんでした\n");
202+ jScrollPane404msg.setViewportView(jTxt404msg);
203+
204+ javax.swing.GroupLayout jPanelTab2Layout = new javax.swing.GroupLayout(jPanelTab2);
205+ jPanelTab2.setLayout(jPanelTab2Layout);
206+ jPanelTab2Layout.setHorizontalGroup(
207+ jPanelTab2Layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
208+ .addComponent(jScrollPane404msg)
209+ .addGroup(javax.swing.GroupLayout.Alignment.TRAILING, jPanelTab2Layout.createSequentialGroup()
210+ .addContainerGap()
211+ .addComponent(jScrollPaneLabel, javax.swing.GroupLayout.DEFAULT_SIZE, 359, Short.MAX_VALUE)
212+ .addContainerGap())
213+ );
214+ jPanelTab2Layout.setVerticalGroup(
215+ jPanelTab2Layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
216+ .addGroup(jPanelTab2Layout.createSequentialGroup()
217+ .addComponent(jScrollPaneLabel, javax.swing.GroupLayout.PREFERRED_SIZE, 38, javax.swing.GroupLayout.PREFERRED_SIZE)
218+ .addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.UNRELATED)
219+ .addComponent(jScrollPane404msg))
220+ );
221+
222+ jTabbedPane1.addTab("結果無し判定", jPanelTab2);
223+
224+ jPanelRtn.setBorder(javax.swing.BorderFactory.createTitledBorder("検索結果"));
225+
226+ jTxtRtn.setColumns(20);
227+ jTxtRtn.setRows(5);
228+ jScrollPaneRtn.setViewportView(jTxtRtn);
229+
230+ javax.swing.GroupLayout jPanelRtnLayout = new javax.swing.GroupLayout(jPanelRtn);
231+ jPanelRtn.setLayout(jPanelRtnLayout);
232+ jPanelRtnLayout.setHorizontalGroup(
233+ jPanelRtnLayout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
234+ .addComponent(jScrollPaneRtn)
235+ );
236+ jPanelRtnLayout.setVerticalGroup(
237+ jPanelRtnLayout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
238+ .addComponent(jScrollPaneRtn, javax.swing.GroupLayout.DEFAULT_SIZE, 163, Short.MAX_VALUE)
239+ );
240+
241+ jMenu1.setText("ファイル");
242+
243+ jMenuLoad.setText("LOAD");
244+ jMenuLoad.addActionListener(new java.awt.event.ActionListener() {
245+ public void actionPerformed(java.awt.event.ActionEvent evt) {
246+ jMenuLoadActionPerformed(evt);
247+ }
248+ });
249+ jMenu1.add(jMenuLoad);
250+
251+ jMenuSave.setText("SAVE");
252+ jMenuSave.addActionListener(new java.awt.event.ActionListener() {
253+ public void actionPerformed(java.awt.event.ActionEvent evt) {
254+ jMenuSaveActionPerformed(evt);
255+ }
256+ });
257+ jMenu1.add(jMenuSave);
258+
259+ jMenuBar1.add(jMenu1);
260+
261+ jMenu3.setText("ツール");
262+
263+ jMenuItem1.setText("ブラウザで表示");
264+ jMenuItem1.addActionListener(new java.awt.event.ActionListener() {
265+ public void actionPerformed(java.awt.event.ActionEvent evt) {
266+ jMenuItem1ActionPerformed(evt);
267+ }
268+ });
269+ jMenu3.add(jMenuItem1);
270+
271+ jMenuBar1.add(jMenu3);
272+
273+ jMenu2.setText("検索");
274+ jMenu2.addMouseListener(new java.awt.event.MouseAdapter() {
275+ public void mouseClicked(java.awt.event.MouseEvent evt) {
276+ jMenu2MouseClicked(evt);
277+ }
278+ });
279+ jMenuBar1.add(jMenu2);
280+
281+ setJMenuBar(jMenuBar1);
282+
283+ javax.swing.GroupLayout layout = new javax.swing.GroupLayout(getContentPane());
284+ getContentPane().setLayout(layout);
285+ layout.setHorizontalGroup(
286+ layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
287+ .addComponent(jPanelRtn, javax.swing.GroupLayout.DEFAULT_SIZE, javax.swing.GroupLayout.DEFAULT_SIZE, Short.MAX_VALUE)
288+ .addGroup(layout.createSequentialGroup()
289+ .addComponent(jLabel1)
290+ .addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.RELATED)
291+ .addComponent(jTxtUrl)
292+ .addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.RELATED)
293+ .addComponent(jBtnSearch))
294+ .addComponent(jTabbedPane1)
295+ );
296+ layout.setVerticalGroup(
297+ layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
298+ .addGroup(layout.createSequentialGroup()
299+ .addGroup(layout.createParallelGroup(javax.swing.GroupLayout.Alignment.BASELINE)
300+ .addComponent(jLabel1)
301+ .addComponent(jTxtUrl, javax.swing.GroupLayout.PREFERRED_SIZE, javax.swing.GroupLayout.DEFAULT_SIZE, javax.swing.GroupLayout.PREFERRED_SIZE)
302+ .addComponent(jBtnSearch))
303+ .addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.RELATED)
304+ .addComponent(jTabbedPane1, javax.swing.GroupLayout.PREFERRED_SIZE, 250, javax.swing.GroupLayout.PREFERRED_SIZE)
305+ .addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.RELATED)
306+ .addComponent(jPanelRtn, javax.swing.GroupLayout.DEFAULT_SIZE, javax.swing.GroupLayout.DEFAULT_SIZE, Short.MAX_VALUE)
307+ .addContainerGap())
308+ );
309+
310+ pack();
311+ }// </editor-fold>//GEN-END:initComponents
312+
313+ private void jBtnRowInsActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_jBtnRowInsActionPerformed
314+ int SelectedRow = jTable1.getSelectedRow();
315+ SearchData sdata = new SearchData();
316+ if(SelectedRow >= 0) {
317+ sdatatblmodel.insertRow(SelectedRow, sdata);
318+ } else {
319+ sdatatblmodel.addRow(sdata);
320+ }
321+ }//GEN-LAST:event_jBtnRowInsActionPerformed
322+
323+ private void jBtnRowDelActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_jBtnRowDelActionPerformed
324+ int SelectedRow = jTable1.getSelectedRow();
325+ if(!(SelectedRow < 0)) {
326+ sdatatblmodel.removeRow(SelectedRow);
327+ }
328+ }//GEN-LAST:event_jBtnRowDelActionPerformed
329+
330+ private void jMenuLoadActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_jMenuLoadActionPerformed
331+ jFileChooser1.setDialogTitle("読込");
332+ int selected = jFileChooser1.showOpenDialog(this);
333+ if (selected == JFileChooser.APPROVE_OPTION) {
334+ File file = jFileChooser1.getSelectedFile();
335+ xmlwriter.load(file);
336+ jTxtUrl.setText(xmlwriter.getTestUrl());
337+ xmlwriter.getSdata();
338+ sdatatblmodel.setRowCount(0);
339+ for(int i = 0; i < SearchData.size(); i++) {
340+ SearchData sdata = SearchData.get(i);
341+ sdatatblmodel.addRow(sdata);
342+ }
343+ }
344+ }//GEN-LAST:event_jMenuLoadActionPerformed
345+
346+ private void jMenuSaveActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_jMenuSaveActionPerformed
347+ jFileChooser1.setDialogTitle("保存");
348+ int selected = jFileChooser1.showSaveDialog(this);
349+ if (selected == JFileChooser.APPROVE_OPTION) {
350+ File file = jFileChooser1.getSelectedFile();
351+ xmlwriter.setTestUrl(jTxtUrl.getText());
352+
353+ SearchData.clear();
354+ for(int row = 0; row < sdatatblmodel.getRowCount(); row++) {
355+ SearchData sdata = sdatatblmodel.getSearchData(row);
356+ SearchData.add(sdata);
357+ }
358+ xmlwriter.setSdata();
359+ xmlwriter.save(file);
360+ }
361+ }//GEN-LAST:event_jMenuSaveActionPerformed
362+
363+ private void jBtnRowCpyActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_jBtnRowCpyActionPerformed
364+ int SelectedRow = jTable1.getSelectedRow();
365+ if(SelectedRow >= 0) {
366+ SearchData sdata = sdatatblmodel.getSearchData(SelectedRow);
367+ sdatatblmodel.insertRow(SelectedRow, sdata);
368+ }
369+ }//GEN-LAST:event_jBtnRowCpyActionPerformed
370+
371+ private void jMenuItem1ActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_jMenuItem1ActionPerformed
372+ Desktop desktop = Desktop.getDesktop();
373+ String uriString = jTxtUrl.getText();
374+ try {
375+ URI uri = new URI(uriString);
376+ desktop.browse(uri);
377+
378+ } catch (URISyntaxException | IOException ex) {
379+ Logger.getLogger(HtmlSearch.class.getName()).log(Level.SEVERE, null, ex);
380+ }
381+ }//GEN-LAST:event_jMenuItem1ActionPerformed
382+
383+ private void jMenu2MouseClicked(java.awt.event.MouseEvent evt) {//GEN-FIRST:event_jMenu2MouseClicked
384+ Search_execution();
385+ }//GEN-LAST:event_jMenu2MouseClicked
386+
387+ private void jBtnSearchActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_jBtnSearchActionPerformed
388+ Search_execution();
389+ }//GEN-LAST:event_jBtnSearchActionPerformed
390+
391+ /**
392+ * 検索実行.
393+ */
394+ void Search_execution() {
395+ jTxtRtn.setText(null);
396+ HtmlParser par = new HtmlParser(jTxtUrl.getText());
397+
398+ // データ無し(404)判定
399+ String strdata = par.getStringPageData();
400+ if(strdata == null) {
401+ jTxtRtn.append("読込みページがありません");
402+ return;
403+ }
404+ String text = jTxt404msg.getText();
405+ String[] strsearch = text.split("\n");
406+ for(String strsearch1 : strsearch) {
407+ if(strdata.contains(strsearch1)) {
408+ jTxtRtn.append(strsearch1);
409+ return;
410+ }
411+ }
412+
413+ // 検索結果
414+ for(int row = 0; row < sdatatblmodel.getRowCount(); row++) {
415+ SearchData sdata = sdatatblmodel.getSearchData(row);
416+ String ans = sdata.getitem();
417+ String rtn = par.search(sdata);
418+ jTxtRtn.append(ans + "\t" + rtn + "\n");
419+ }
420+
421+ jTxtRtn.setCaretPosition(0);
422+ }
423+
424+ /**
425+ * @param args the command line arguments
426+ */
427+ public static void main(String args[]) {
428+ /* Set the Nimbus look and feel */
429+ //<editor-fold defaultstate="collapsed" desc=" Look and feel setting code (optional) ">
430+ /* If Nimbus (introduced in Java SE 6) is not available, stay with the default look and feel.
431+ * For details see http://download.oracle.com/javase/tutorial/uiswing/lookandfeel/plaf.html
432+ */
433+ try {
434+ for (javax.swing.UIManager.LookAndFeelInfo info : javax.swing.UIManager.getInstalledLookAndFeels()) {
435+ if ("Nimbus".equals(info.getName())) {
436+ javax.swing.UIManager.setLookAndFeel(info.getClassName());
437+ break;
438+ }
439+ }
440+ } catch (ClassNotFoundException
441+ | InstantiationException
442+ | IllegalAccessException
443+ | javax.swing.UnsupportedLookAndFeelException ex) {
444+ java.util.logging.Logger.getLogger(HtmlSearch.class.getName()).log(java.util.logging.Level.SEVERE, null, ex);
445+ }
446+ //</editor-fold>
447+
448+ /* Create and display the form */
449+ java.awt.EventQueue.invokeLater(new Runnable() {
450+ @Override
451+ public void run() {
452+ new HtmlSearch().setVisible(true);
453+ }
454+ });
455+ }
456+
457+ // Variables declaration - do not modify//GEN-BEGIN:variables
458+ private javax.swing.JButton jBtnRowCpy;
459+ private javax.swing.JButton jBtnRowDel;
460+ private javax.swing.JButton jBtnRowIns;
461+ private javax.swing.JButton jBtnSearch;
462+ private javax.swing.JFileChooser jFileChooser1;
463+ private javax.swing.JLabel jLabel1;
464+ private javax.swing.JMenu jMenu1;
465+ private javax.swing.JMenu jMenu2;
466+ private javax.swing.JMenu jMenu3;
467+ private javax.swing.JMenuBar jMenuBar1;
468+ private javax.swing.JMenuItem jMenuItem1;
469+ private javax.swing.JMenuItem jMenuLoad;
470+ private javax.swing.JMenuItem jMenuSave;
471+ private javax.swing.JPanel jPanelRtn;
472+ private javax.swing.JPanel jPanelTab1;
473+ private javax.swing.JPanel jPanelTab2;
474+ private javax.swing.JRadioButton jRadioButton1;
475+ private javax.swing.JScrollPane jScrollPane1;
476+ private javax.swing.JScrollPane jScrollPane404msg;
477+ private javax.swing.JScrollPane jScrollPaneLabel;
478+ private javax.swing.JScrollPane jScrollPaneRtn;
479+ private javax.swing.JTabbedPane jTabbedPane1;
480+ private javax.swing.JTable jTable1;
481+ private javax.swing.JTextArea jTxt404msg;
482+ private javax.swing.JTextArea jTxtLabel;
483+ private javax.swing.JTextArea jTxtRtn;
484+ private javax.swing.JTextField jTxtUrl;
485+ // End of variables declaration//GEN-END:variables
486+}
487+
488+class SearchDataTableModel extends DefaultTableModel {
489+ /* ---------------------------------------------------------------------- *
490+ * データ属性
491+ * ---------------------------------------------------------------------- */
492+ public String[] columnName = {
493+ /* 0 */ "項目名",
494+ /* 1 */ "タグ",
495+ /* 2 */ "ＩＤ",
496+ /* 3 */ "クラス",
497+ /* 4 */ "位置",
498+ /* 5 */ "抽出条件"
499+ };
500+
501+ public Class[] columnClass = {
502+ /* 0 */ String.class,
503+ /* 1 */ String.class,
504+ /* 2 */ String.class,
505+ /* 3 */ String.class,
506+ /* 4 */ String.class,
507+ /* 5 */ String.class
508+ };
509+
510+ int column_item = 0;
511+ int column_htmltag = 1;
512+ int column_htmlid = 2;
513+ int column_htmlclass = 3;
514+ int column_around = 4;
515+ int column_regexp = 5;
516+
517+ /* ---------------------------------------------------------------------- *
518+ * 処理
519+ * ---------------------------------------------------------------------- */
520+ @Override
521+ public String getColumnName(int modelIndex) {
522+ return columnName[modelIndex];
523+ }
524+
525+ @Override
526+ public Class<?> getColumnClass(int modelIndex) {
527+ return columnClass[modelIndex];
528+ }
529+
530+ @Override
531+ public int getColumnCount() {
532+ return columnName.length;
533+ }
534+
535+ /* ---------------------------------------------------------------------- */
536+
537+ public SearchData getSearchData(int row) {
538+ SearchData sdata = new SearchData();
539+ sdata.setitem(String.valueOf(getValueAt(row, column_item)));
540+ sdata.setHtmltag(String.valueOf(getValueAt(row, column_htmltag)));
541+ sdata.setHtmlid(String.valueOf(getValueAt(row, column_htmlid)));
542+ sdata.setHtmlclass(String.valueOf(getValueAt(row, column_htmlclass)));
543+ sdata.setaround(String.valueOf(getValueAt(row, column_around)));
544+ sdata.setregexp(String.valueOf(getValueAt(row, column_regexp)));
545+ return sdata;
546+ }
547+
548+ public void addRow(SearchData sdata) {
549+ addRow(getObjdata(sdata));
550+ }
551+
552+ public void insertRow(int row, SearchData sdata) {
553+ insertRow(row, getObjdata(sdata));
554+ }
555+
556+ private Object[] getObjdata(SearchData sdata) {
557+ Object[] obj = new Object[] {
558+ sdata.getitem(),
559+ sdata.getHtmltag(),
560+ sdata.getHtmlid(),
561+ sdata.getHtmlclass(),
562+ sdata.getaround(),
563+ sdata.getregexp()
564+ };
565+ return obj;
566+ }
567+
568+}
\ No newline at end of file
Added: svn:keywords
## -0,0 +1 ##
+Id
\ No newline at end of property
--- branches/b4/webScraping/src/WebScraping/core/AttributeData.java (nonexistent)
+++ branches/b4/webScraping/src/WebScraping/core/AttributeData.java (revision 134)
@@ -0,0 +1,164 @@
1+/*
2+ * Copyright (C) 2014 kgto.
3+ *
4+ * This library is free software; you can redistribute it and/or
5+ * modify it under the terms of the GNU Lesser General Public
6+ * License as published by the Free Software Foundation; either
7+ * version 2.1 of the License, or (at your option) any later version.
8+ *
9+ * This library is distributed in the hope that it will be useful,
10+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
11+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
12+ * Lesser General Public License for more details.
13+ *
14+ * You should have received a copy of the GNU Lesser General Public
15+ * License along with this library; if not, write to the Free Software
16+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17+ * MA 02110-1301 USA
18+ */
19+/*
20+ * $Id$
21+ */
22+
23+package webScraping.core;
24+
25+import java.util.ArrayList;
26+import java.util.Enumeration;
27+import javax.swing.text.MutableAttributeSet;
28+import javax.swing.text.html.HTML;
29+
30+/**
31+ * ＨＴＭＬタグの属性情報を保持する.
32+ * @author kgto
33+ */
34+public class AttributeData {
35+
36+ public AttributeData() {
37+ AttrList = new ArrayList();
38+ size = 0;
39+ }
40+
41+ /**
42+ * 属性情報追加.
43+ * @param tag
44+ * @param attr
45+ */
46+ public void add(HTML.Tag tag, MutableAttributeSet attr) {
47+
48+ int tagcount = tagcnt(tag);
49+ ++tagcount;
50+
51+ Enumeration e = attr.getAttributeNames();
52+ while(e.hasMoreElements()) {
53+ Object obj = e.nextElement();
54+
55+ AttrData a = new AttrData();
56+ a.tag = tag;
57+ a.count = tagcount;
58+ a.attrname = obj.toString();
59+ a.attrvalue = attr.getAttribute(obj).toString();
60+
61+ AttrList.add(a);
62+ size = AttrList.size();
63+ }
64+
65+ }
66+
67+ /**
68+ * 属性情報検索.
69+ * @param tag
70+ * @param attrname
71+ * @param attrvalue
72+ * @return
73+ */
74+ public boolean search(HTML.Tag tag, String attrname, String attrvalue) {
75+ boolean ret = false;
76+ for (Object AttrList1 : AttrList) {
77+ AttrData a = (AttrData)AttrList1;
78+ if(a.tag == tag) {
79+ //if(a.attrname.equals(attrname) && a.attrvalue.equals(attrvalue)) {
80+ if(a.attrname.equals(attrname) && a.attrvalue.startsWith(attrvalue)) {
81+ ret = true;
82+ }
83+ }
84+ }
85+ return ret;
86+ }
87+
88+ public boolean searchId(HTML.Tag tag, String attrvalue) {
89+ return search(tag, "id", attrvalue);
90+ }
91+
92+ public boolean searchClass(HTML.Tag tag, String attrvalue) {
93+ return search(tag, "class", attrvalue);
94+ }
95+
96+ /**
97+ * 属性の値を取得する.
98+ * @param tag
99+ * @param attrname
100+ * @return
101+ */
102+ public ArrayList getvale(HTML.Tag tag, String attrname) {
103+ ArrayList ret = new ArrayList();
104+ for (Object AttrList1 : AttrList) {
105+ AttrData a = (AttrData)AttrList1;
106+ if(a.tag == tag) {
107+ if(a.attrname.equals(attrname)) {
108+ ret.add(a.attrvalue);
109+ }
110+ }
111+ }
112+ return ret;
113+ }
114+
115+ /**
116+ * 引数で渡されたTAGの最新カウント数を返す.
117+ * @param tag
118+ * @return
119+ */
120+ private int tagcnt(HTML.Tag tag) {
121+ int wkcnt = 0;
122+ for (Object AttrList1 : AttrList) {
123+ AttrData a = (AttrData)AttrList1;
124+ if(a.tag == tag) {
125+ if(wkcnt < a.count) {
126+ wkcnt = a.count;
127+ }
128+ }
129+ }
130+ return wkcnt;
131+ }
132+
133+ // AttrList の内容を返すメソッド
134+ public HTML.Tag gettag(int i) {
135+ AttrData a = (AttrData)AttrList.get(i);
136+ return a.tag;
137+ }
138+
139+ public int getcount(int i) {
140+ AttrData a = (AttrData)AttrList.get(i);
141+ return a.count;
142+ }
143+
144+ public String getattrname(int i) {
145+ AttrData a = (AttrData)AttrList.get(i);
146+ return a.attrname;
147+ }
148+
149+ public String getattrvalue(int i) {
150+ AttrData a = (AttrData)AttrList.get(i);
151+ return a.attrvalue;
152+ }
153+
154+ // フィールド変数
155+ public class AttrData {
156+ public HTML.Tag tag;
157+ public int count;
158+ public String attrname;
159+ public String attrvalue;
160+ }
161+ public ArrayList AttrList;
162+ public int size; // AttrListのサイズ
163+
164+}
Added: svn:keywords
## -0,0 +1 ##
+Id
\ No newline at end of property
--- branches/b4/webScraping/src/WebScraping/core/HtmlParserCallback.java (nonexistent)
+++ branches/b4/webScraping/src/WebScraping/core/HtmlParserCallback.java (revision 134)
@@ -0,0 +1,222 @@
1+/*
2+ * Copyright (C) 2014 kgto.
3+ *
4+ * This library is free software; you can redistribute it and/or
5+ * modify it under the terms of the GNU Lesser General Public
6+ * License as published by the Free Software Foundation; either
7+ * version 2.1 of the License, or (at your option) any later version.
8+ *
9+ * This library is distributed in the hope that it will be useful,
10+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
11+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
12+ * Lesser General Public License for more details.
13+ *
14+ * You should have received a copy of the GNU Lesser General Public
15+ * License along with this library; if not, write to the Free Software
16+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17+ * MA 02110-1301 USA
18+ */
19+/*
20+ * $Id$
21+ */
22+
23+package webScraping.core;
24+
25+import java.util.ArrayList;
26+import java.util.HashMap;
27+import javax.swing.text.MutableAttributeSet;
28+import javax.swing.text.html.HTML;
29+import javax.swing.text.html.HTMLEditorKit;
30+
31+/**
32+ * ＨＴＭＬパーサ部品.
33+ * @author kgto
34+ */
35+class HtmlParserCallback extends HTMLEditorKit.ParserCallback {
36+ /* ---------------------------------------------------------------------- *
37+ * フィールド
38+ * ---------------------------------------------------------------------- */
39+ // Tag毎の階層
40+ HashMap<HTML.Tag,Integer> tagMap = new HashMap<>();
41+
42+ // serach key 情報
43+ String keytag;
44+ String keyid;
45+ String keyclass;
46+
47+ // serach key と一致時の情報退避
48+ int bufCount = 0;
49+ HTML.Tag bufTag = null;
50+ // serach key と一致時の情報格納ワーク
51+ StringBuilder bufText;
52+
53+ // serach key と一致時のデータ一覧
54+ ArrayList sData;
55+
56+ // 属性データ
57+ AttributeData attrdata;
58+
59+ /* ---------------------------------------------------------------------- *
60+ * コンストラクタ
61+ * ---------------------------------------------------------------------- */
62+ protected HtmlParserCallback(SearchData skey) {
63+
64+ // キー情報展開
65+ keytag = skey.getHtmltag();
66+ keyid = skey.getHtmlid();
67+ keyclass = skey.getHtmlclass();
68+
69+ sData = new ArrayList();
70+ }
71+
72+ /* ---------------------------------------------------------------------- *
73+ * Getter
74+ * ---------------------------------------------------------------------- */
75+ ArrayList getrtnData() {
76+ return this.sData;
77+ }
78+
79+ /* ---------------------------------------------------------------------- *
80+ * メソッド
81+ * ---------------------------------------------------------------------- */
82+ @Override
83+ public void handleStartTag(HTML.Tag tag, MutableAttributeSet attr, int pos){
84+ // Tag毎の階層を保持
85+ int count = 1;
86+ if(tagMap.containsKey(tag)) {
87+ count = tagMap.get(tag);
88+ count++;
89+ }
90+ tagMap.put(tag, count);
91+
92+ // 属性解析
93+ AttributeData handleStartattrdata = new AttributeData();
94+ handleStartattrdata.add(tag, attr);
95+
96+ DebugProcess.htmlinfo(tag, attr, "handleStartTag", count);
97+
98+ if(bufCount == 0) {
99+ if(tag.toString().equals(keytag)) {
100+ //if(serachAttribute(attr)) {
101+ if(serachAttribute(tag, handleStartattrdata)) {
102+ bufCount = count;
103+ bufTag = tag;
104+ attrdata = new AttributeData();
105+ bufText = new StringBuilder();
106+ }
107+ }
108+ }
109+ if(bufCount > 0) {
110+ attrdata.add(tag, attr);
111+ }
112+ }
113+
114+ @Override
115+ public void handleEndTag(HTML.Tag tag, int pos){
116+ // Tag毎の階層を取得
117+ int count = 0;
118+ if(tagMap.containsKey(tag)) {
119+ count = tagMap.get(tag);
120+ }
121+
122+ DebugProcess.htmlinfo(tag, null, "handleEndTag", count);
123+
124+ if(tag.equals(bufTag) && count <= bufCount) {
125+
126+ // 溜め込んだ一致情報をリストへ格納
127+ sData.add(bufText.toString());
128+
129+ // 退避したserach keyとの一致情報クリア
130+ bufCount = 0;
131+ bufTag = null;
132+ bufText = null;
133+ }
134+
135+ // Tag毎の階層減算
136+ tagMap.put(tag, --count);
137+ }
138+
139+ @Override
140+ public void handleText(char[] data, int pos){
141+
142+ DebugProcess.htmlinfo(data, "handleText");
143+
144+ String splitchar = "\t";
145+ //制御文字の削除
146+ // &nbsp; 0xa0
147+ StringBuilder buf = new StringBuilder();
148+ for(int i = 0; i < data.length; i++) {
149+ if(data[i] > 0x1f && data[i] != 0x7f && data[i] != 0xa0) {
150+ buf.append(data[i]);
151+ }
152+ }
153+ if(bufCount > 0) {
154+ if(bufText.length() > 0) {
155+ bufText.append(splitchar);
156+ }
157+ bufText.append(buf.toString());
158+ }
159+ }
160+
161+ @Override
162+ public void handleSimpleTag(HTML.Tag tag, MutableAttributeSet attr, int pos){
163+ if(bufCount > 0) {
164+ attrdata.add(tag, attr);
165+ }
166+ DebugProcess.htmlinfo(tag, attr, "handleSimpleTag", 0);
167+ }
168+
169+ /**
170+ * ページ内のID/CLASS値と検索キーを比較する.
171+ * @param attr ページのMutableAttributeSet
172+ * @return boolean 検索キーと一致の時、true
173+ */
174+ boolean serachAttribute(MutableAttributeSet attr) {
175+ String currentID = (String)attr.getAttribute(HTML.Attribute.ID);
176+ String currentClass = (String)attr.getAttribute(HTML.Attribute.CLASS);
177+
178+ if(keyid.isEmpty() == false && keyclass.isEmpty() == false) {
179+ if(keyid.equals(currentID) && keyclass.equals(currentClass)) {
180+ return true;
181+ }
182+ }
183+
184+ if(keyid.isEmpty() == false) {
185+ if(keyid.equals(currentID)) {
186+ return true;
187+ }
188+ }
189+
190+ if(keyclass.isEmpty() == false) {
191+ if(keyclass.equals(currentClass)) {
192+ return true;
193+ }
194+ }
195+
196+ return false;
197+ }
198+
199+ /**
200+ * ページ内のID/CLASS値と検索キーを比較する.
201+ * @param tag
202+ * @param attrdata
203+ * @return boolean 検索キーと一致の時、true
204+ */
205+ boolean serachAttribute(HTML.Tag tag, AttributeData attrdata) {
206+ // ID と CLASS の両方にキー入力有りの場合
207+ if(keyid.isEmpty() == false && keyclass.isEmpty() == false) {
208+ if(attrdata.searchId(tag, keyid) && attrdata.searchClass(tag, keyclass)) {
209+ return true;
210+ }
211+ }
212+ // ID のキーチェック
213+ if(keyid.isEmpty() == false) {
214+ return attrdata.searchId(tag, keyid);
215+ }
216+ // CLASS のキーチェック
217+ if(keyclass.isEmpty() == false) {
218+ return attrdata.searchClass(tag, keyclass);
219+ }
220+ return false;
221+ }
222+}
Added: svn:keywords
## -0,0 +1 ##
+Id
\ No newline at end of property
--- branches/b4/webScraping/src/WebScraping/core/SearchData.java (nonexistent)
+++ branches/b4/webScraping/src/WebScraping/core/SearchData.java (revision 134)
@@ -0,0 +1,200 @@
1+/*
2+ * Copyright (C) 2014 kgto.
3+ *
4+ * This library is free software; you can redistribute it and/or
5+ * modify it under the terms of the GNU Lesser General Public
6+ * License as published by the Free Software Foundation; either
7+ * version 2.1 of the License, or (at your option) any later version.
8+ *
9+ * This library is distributed in the hope that it will be useful,
10+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
11+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
12+ * Lesser General Public License for more details.
13+ *
14+ * You should have received a copy of the GNU Lesser General Public
15+ * License along with this library; if not, write to the Free Software
16+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17+ * MA 02110-1301 USA
18+ */
19+/*
20+ * $Id$
21+ */
22+
23+package webScraping.core;
24+
25+import java.util.ArrayList;
26+
27+/**
28+ * タグ検索データ.
29+ * @author kgto
30+ */
31+public class SearchData {
32+ /* ---------------------------------------------------------------------- *
33+ * フィールド
34+ * ---------------------------------------------------------------------- */
35+ private String item;
36+ private String htmltag;
37+ private String htmlid;
38+ private String htmlclass;
39+ private String around;
40+ private String regexp;
41+
42+ /* ---------------------------------------------------------------------- *
43+ * static 処理
44+ * ---------------------------------------------------------------------- */
45+ public static class Context {
46+ public Class columnClass;
47+ public String columnName;
48+ public String columnNameJp;
49+
50+ public Context(Class columnClass, String columnName, String columnNameJp) {
51+ this.columnClass = columnClass;
52+ this.columnName = columnName;
53+ this.columnNameJp = columnNameJp;
54+ }
55+ }
56+
57+ public static final Context[] context = {
58+ /* 0 */ new Context(String.class , "item" , "項目名"),
59+ /* 1 */ new Context(String.class , "htmltag" , "タグ"),
60+ /* 2 */ new Context(String.class , "htmlid" , "ＩＤ"),
61+ /* 3 */ new Context(String.class , "htmlclass" , "クラス"),
62+ /* 4 */ new Context(String.class , "around" , "位置"),
63+ /* 5 */ new Context(String.class , "regexp" , "抽出条件")
64+ };
65+
66+ /* ---------------------------------------------------------------------- */
67+ private static ArrayList<SearchData> slist = new ArrayList<>();
68+
69+ public static void addSearchData(
70+ String item, String htmltag, String htmlid,
71+ String htmlclass, String around, String regexp) {
72+ SearchData sdat = new SearchData();
73+ sdat.setitem(item);
74+ sdat.setHtmltag(htmltag);
75+ sdat.setHtmlid(htmlid);
76+ sdat.setHtmlclass(htmlclass);
77+ sdat.setaround(around);
78+ sdat.setregexp(regexp);
79+
80+ slist.add(sdat);
81+ }
82+
83+ public static void add(SearchData sdat) {
84+ slist.add(sdat);
85+ }
86+
87+ public static SearchData get(int i) {
88+ return slist.get(i);
89+ }
90+
91+ public static int size() {
92+ return slist.size();
93+ }
94+
95+ public static SearchData remove(int index) {
96+ return slist.remove(index);
97+ }
98+
99+ public static void clear() {
100+ slist.clear();
101+ }
102+
103+ /* ---------------------------------------------------------------------- *
104+ * コンストラクタ
105+ * ---------------------------------------------------------------------- */
106+ public SearchData() {
107+ initialize();
108+ }
109+
110+ public SearchData(SearchData dat) {
111+ this.item = dat.getitem();
112+ this.htmltag = dat.getHtmltag();
113+ this.htmlid = dat.getHtmlid();
114+ this.htmlclass = dat.getHtmlclass();
115+ this.around = dat.getaround();
116+ this.regexp = dat.getregexp();
117+ }
118+
119+ /* ---------------------------------------------------------------------- *
120+ * Setter
121+ * ---------------------------------------------------------------------- */
122+ public void setitem(String item) {
123+ this.item = item;
124+ }
125+
126+ public void setHtmltag(String htmltag) {
127+ this.htmltag = htmltag;
128+ }
129+
130+ public void setHtmlid(String htmlid) {
131+ this.htmlid = htmlid;
132+ }
133+
134+ public void setHtmlclass(String htmlclass) {
135+ this.htmlclass = htmlclass;
136+ }
137+
138+ public void setaround(String around) {
139+ this.around = around;
140+ }
141+
142+ public void setregexp(String regexp) {
143+ this.regexp = regexp;
144+ }
145+
146+ /* ---------------------------------------------------------------------- *
147+ * Getter
148+ * ---------------------------------------------------------------------- */
149+ public String getitem() {
150+ return item;
151+ }
152+
153+ public String getHtmltag() {
154+ return htmltag;
155+ }
156+
157+ public String getHtmlid() {
158+ return htmlid;
159+ }
160+
161+ public String getHtmlclass() {
162+ return htmlclass;
163+ }
164+
165+ public String getaround() {
166+ return around;
167+ }
168+
169+ public String getregexp() {
170+ return regexp;
171+ }
172+
173+ /* ---------------------------------------------------------------------- *
174+ * メソッド
175+ * ---------------------------------------------------------------------- */
176+ /**
177+ * データ初期化.
178+ */
179+ public final void initialize() {
180+ this.item = "";
181+ this.htmltag = "";
182+ this.htmlid = "";
183+ this.htmlclass = "";
184+ this.around = "";
185+ this.regexp = "";
186+ }
187+
188+ public Object[] getObjData() {
189+ Object[] obj = {
190+ /* 0 */ getitem(), // 項目名
191+ /* 1 */ getHtmltag(), // タグ
192+ /* 2 */ getHtmlid(), // ＩＤ
193+ /* 3 */ getHtmlclass(), // クラス
194+ /* 4 */ getaround(), // 位置
195+ /* 5 */ getregexp() // 抽出条件
196+ };
197+ return obj;
198+ }
199+
200+}
Added: svn:keywords
## -0,0 +1 ##
+Id
\ No newline at end of property
--- branches/b4/webScraping/src/WebScraping/core/HtmlParser.java (nonexistent)
+++ branches/b4/webScraping/src/WebScraping/core/HtmlParser.java (revision 134)
@@ -0,0 +1,273 @@
1+/*
2+ * Copyright (C) 2014 kgto.
3+ *
4+ * This library is free software; you can redistribute it and/or
5+ * modify it under the terms of the GNU Lesser General Public
6+ * License as published by the Free Software Foundation; either
7+ * version 2.1 of the License, or (at your option) any later version.
8+ *
9+ * This library is distributed in the hope that it will be useful,
10+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
11+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
12+ * Lesser General Public License for more details.
13+ *
14+ * You should have received a copy of the GNU Lesser General Public
15+ * License along with this library; if not, write to the Free Software
16+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17+ * MA 02110-1301 USA
18+ */
19+/*
20+ * $Id$
21+ */
22+
23+package webScraping.core;
24+
25+import java.io.*;
26+import java.net.*;
27+import java.util.ArrayList;
28+import java.util.logging.Level;
29+import java.util.logging.Logger;
30+import java.util.regex.Matcher;
31+import java.util.regex.Pattern;
32+import javax.swing.text.html.parser.ParserDelegator;
33+
34+/**
35+ * ＨＴＭＬパーサ.
36+ * @author kgto
37+ */
38+public class HtmlParser {
39+ /* ---------------------------------------------------------------------- *
40+ * フィールド
41+ * ---------------------------------------------------------------------- */
42+ URL url;
43+ String pageData;
44+ ArrayList sData;
45+
46+ // 作業ワーク
47+ private String htmltag;
48+ private String htmlid;
49+ private String htmlclass;
50+
51+ /* ---------------------------------------------------------------------- *
52+ * コンストラクタ
53+ * ---------------------------------------------------------------------- */
54+ public HtmlParser(URL UrlAdress) {
55+ DebugProcess.debuglog_set();
56+ this.url = UrlAdress;
57+ getPageData();
58+ }
59+
60+ public HtmlParser(String UrlAdress) {
61+ DebugProcess.debuglog_set();
62+ try {
63+ url = new URL(UrlAdress);
64+ getPageData();
65+
66+ } catch (MalformedURLException ex) {
67+ Logger.getLogger(HtmlParser.class.getName()).log(Level.SEVERE, null, ex);
68+ }
69+ }
70+
71+ public HtmlParser() {
72+ DebugProcess.debuglog_set();
73+ url = null;
74+ }
75+
76+ /* ---------------------------------------------------------------------- *
77+ * Getter
78+ * ---------------------------------------------------------------------- */
79+ public String getStringPageData() {
80+ return pageData;
81+ }
82+
83+ /* ---------------------------------------------------------------------- *
84+ * Setter
85+ * ---------------------------------------------------------------------- */
86+ public void seturl(URL UrlAdress) {
87+ this.url = UrlAdress;
88+ getPageData();
89+ }
90+
91+ /* ---------------------------------------------------------------------- *
92+ * メソッド
93+ * ---------------------------------------------------------------------- */
94+ public void seturl(String UrlAdress) {
95+ try {
96+ url = new URL(UrlAdress);
97+ getPageData();
98+
99+ } catch (MalformedURLException ex) {
100+ Logger.getLogger(HtmlParser.class.getName()).log(Level.SEVERE, null, ex);
101+ }
102+ }
103+
104+ /**
105+ * ＨＴＭＬページ内検索.
106+ * 検索キーとして渡されたタグ,ID,クラスから、対象となるタグを探し出し、
107+ * around(タグ位置)として指定された箇所の文字列をregexp(正規表現)で指定された整形を
108+ * 行った結果を返す。<br>
109+ * aroundの初期値:0 検索キーとして未指定(未入力)の場合、最初(0)の文字列。<br>
110+ * regexpが指定(入力)ありの場合、正規表現にて整形を行う。<br>
111+ * 渡された検索キーに一致するタグが存在しなかった場合、NULLを返す。
112+ * @param skey 検索キーデータ(SearchData)
113+ * @return String　検索キーに一致するデータの文字列
114+ */
115+ public String search(SearchData skey) {
116+
117+ // htmlページ内を検索
118+ if(isHtmlkeyEq(skey) == false) {
119+ searchPageData(skey);
120+ }
121+ /*
122+ around 出現位置指定 入力有り：指定された位置の情報のみ返す。
123+ 入力無し：取得した全ての情報を返す。
124+ */
125+ String regexp = skey.getregexp();
126+ if(skey.getaround().length() > 0) {
127+ int wkAround = Integer.parseInt(skey.getaround()); // 検索位置を数値変換
128+ if(wkAround < sData.size()) {
129+ String str = (String)sData.get(wkAround);
130+ String rtn = RegularExpression(str, regexp);
131+ return rtn;
132+ }
133+ } else {
134+ StringBuilder strbuf = new StringBuilder();
135+ for (Object sData1 : sData) {
136+ String str = (String)sData1;
137+ String rtn = RegularExpression(str, regexp);
138+ if(strbuf.length() > 0) {
139+ strbuf.append("\t");
140+ }
141+ strbuf.append(rtn);
142+ }
143+ return strbuf.toString();
144+ }
145+ return null;
146+ }
147+
148+ /**
149+ * 直近のHTMLタグ/ID/CLASS値と引数の値を比較する.
150+ * @param skey HTMLタグ/ID/CLASSが格納された検索キー
151+ * @return boolean HTMLタグ/ID/CLASS値が一致する時、true
152+ */
153+ boolean isHtmlkeyEq(SearchData skey) {
154+
155+ String stag = skey.getHtmltag();
156+ String sid = skey.getHtmlid();
157+ String sclass = skey.getHtmlclass();
158+
159+ boolean rtn = true;
160+
161+ // htmltag
162+ if(htmltag == null) {
163+ rtn = false;
164+ } else {
165+ if(htmltag.equals(stag) == false) {
166+ rtn = false;
167+ }
168+ }
169+
170+ // htmlid
171+ if(htmlid == null) {
172+ rtn = false;
173+ } else {
174+ if(htmlid.equals(sid) == false) {
175+ rtn = false;
176+ }
177+ }
178+
179+ // htmlclass
180+ if(htmlclass == null) {
181+ rtn = false;
182+ } else {
183+ if(htmlclass.equals(sclass) == false) {
184+ rtn = false;
185+ }
186+ }
187+
188+ if(!rtn) {
189+ htmltag = stag;
190+ htmlid = sid;
191+ htmlclass = sclass;
192+ }
193+
194+ return rtn;
195+ }
196+
197+ /**
198+ * 正規表現検索.
199+ * @param strdata
200+ * @param regexp
201+ * @return
202+ */
203+ String RegularExpression(String strdata, String regexp) {
204+ String expdata = null;
205+
206+ //regexpのチェック
207+ if(regexp.isEmpty()) {
208+ expdata = strdata;
209+ return expdata;
210+ }
211+
212+ //正規表現検索
213+ Pattern ptn = Pattern.compile(regexp);
214+ Matcher matchdata = ptn.matcher(strdata);
215+ if (matchdata.find()) {
216+ if(matchdata.groupCount() >= 1) {
217+ expdata = matchdata.group(1);
218+ }
219+ }
220+ return expdata;
221+ }
222+
223+ /**
224+ * インターネット接続.
225+ */
226+ private void getPageData() {
227+ HttpURLConnection con = null;
228+ try {
229+ con = (HttpURLConnection)url.openConnection();
230+ con.setRequestMethod("GET");
231+ BufferedReader reader = new BufferedReader(
232+ new InputStreamReader(con.getInputStream(), "utf-8"));
233+ String wkline;
234+ StringBuilder sb = new StringBuilder();
235+ while((wkline = reader.readLine()) != null) {
236+ sb.append(wkline).append("\n");
237+ }
238+ pageData = sb.toString();
239+
240+ } catch(FileNotFoundException ex) {
241+ pageData = null;
242+ } catch (IOException ex) {
243+ Logger.getLogger(HtmlParser.class.getName()).log(Level.SEVERE, null, ex);
244+ } finally {
245+ if(con != null) {
246+ con.disconnect();
247+ }
248+ }
249+ }
250+
251+ /**
252+ * ＨＴＭＬパーサ.
253+ * @param skey
254+ */
255+ private void searchPageData(SearchData skey) {
256+
257+ DebugProcess.searchDatainfo(skey);
258+
259+ Reader reader;
260+ try {
261+ reader = new BufferedReader(new StringReader(pageData));
262+ HtmlParserCallback cb = new HtmlParserCallback(skey);
263+ ParserDelegator pd = new ParserDelegator();
264+ pd.parse(reader, cb, true);
265+ reader.close();
266+
267+ sData = cb.getrtnData();
268+
269+ } catch (IOException ex) {
270+ Logger.getLogger(HtmlParser.class.getName()).log(Level.SEVERE, null, ex);
271+ }
272+ }
273+}
Added: svn:keywords
## -0,0 +1 ##
+Id
\ No newline at end of property
--- branches/b4/webScraping/src/WebScraping/core/DebugProcess.java (nonexistent)
+++ branches/b4/webScraping/src/WebScraping/core/DebugProcess.java (revision 134)
@@ -0,0 +1,264 @@
1+/*
2+ * Copyright (C) 2014 kgto.
3+ *
4+ * This library is free software; you can redistribute it and/or
5+ * modify it under the terms of the GNU Lesser General Public
6+ * License as published by the Free Software Foundation; either
7+ * version 2.1 of the License, or (at your option) any later version.
8+ *
9+ * This library is distributed in the hope that it will be useful,
10+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
11+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
12+ * Lesser General Public License for more details.
13+ *
14+ * You should have received a copy of the GNU Lesser General Public
15+ * License along with this library; if not, write to the Free Software
16+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17+ * MA 02110-1301 USA
18+ */
19+/*
20+ * $Id$
21+ */
22+
23+package webScraping.core;
24+
25+import java.io.File;
26+import java.io.FileInputStream;
27+import java.io.FileNotFoundException;
28+import java.io.IOException;
29+import java.util.logging.FileHandler;
30+import java.util.logging.Formatter;
31+import java.util.logging.Handler;
32+import java.util.logging.Level;
33+import java.util.logging.LogManager;
34+import java.util.logging.LogRecord;
35+import java.util.logging.Logger;
36+import javax.swing.text.MutableAttributeSet;
37+import javax.swing.text.html.HTML;
38+
39+/**
40+ * デバック情報.
41+ * カレントディレクトリに設定ファイル(Debug.prop)を置くことで、デバックログの出力を制御する。
42+ * @author kgto
43+ */
44+public class DebugProcess {
45+ // 設定ファイル名
46+ protected static final String configurationFilename = "Debug.prop";
47+ // ロガー名
48+ protected static final Logger logger = Logger.getLogger("WebScraping");
49+ // ログ出力デフォルトレベル
50+ protected static final Level loggerlevel = Level.FINEST;
51+
52+
53+ /**
54+ * ログ出力設定.
55+ * ログ設定ファイルの存在をチェック、(最終的な)ログレベルにより、
56+ * ファイルハンドラの設定と出力書式の設定を行う。
57+ */
58+ public static void debuglog_set() {
59+ try {
60+ initLogConfiguration();
61+
62+ if(Level.ALL.equals(logger.getLevel())) {
63+ //logger.addHandler(new FileHandler("WebScraping%g.log", 100000, 2));
64+ logger.addHandler(new FileHandler("WebScraping%g.log", true));
65+ }
66+ setFomatter();
67+
68+ } catch (IOException | SecurityException ex) {
69+ Logger.getLogger(DebugProcess.class.getName()).log(Level.SEVERE, null, ex);
70+ }
71+ }
72+
73+ /**
74+ * ログ出力設定解除.
75+ */
76+ public static void debuglog_unset() {
77+ }
78+
79+
80+ /**
81+ * デバック出力(ＨＴＭＬ解析-タグ＆属性).
82+ * ＨＴＭＬのタグと属性の解析状態を出力する。
83+ * 書式: 9 : x : タグ名 [属性名]属性数 = 属性値<br>
84+ * 凡例: 9 = 階層レベル(count値), x = F(tagの開始)/E(tagの終了)/S(単独tag)の何れか１文字<br>
85+ * @param tag タグ
86+ * @param attr 属性
87+ * @param methodname このメソッドを呼び出した親メソッド名
88+ * @param count ＨＴＭＬタグの階層レベル
89+ */
90+ public static void htmlinfo(HTML.Tag tag, MutableAttributeSet attr,
91+ String methodname, int count) {
92+
93+ // ログ出力レベルチェック
94+ if(logger.getLevel() == null) {
95+ return;
96+ }
97+ if(logger.getLevel().intValue() > loggerlevel.intValue()) {
98+ return;
99+ }
100+
101+ // 編集処理
102+ char kbn = ' ';
103+ if("handleStartTag".equals(methodname)) {
104+ kbn = 'F';
105+ }
106+ if("handleEndTag".equals(methodname)) {
107+ kbn = 'E';
108+ }
109+ if("handleSimpleTag".equals(methodname)) {
110+ kbn = 'S';
111+ }
112+
113+ StringBuilder strBuf = new StringBuilder(80);
114+ strBuf.append(count).append(" : ");
115+ strBuf.append(kbn).append(" : ");
116+ strBuf.append(tag.toString());
117+ // 属性情報
118+ if(attr != null) {
119+ if(attr.getAttributeCount() != 0) {
120+ AttributeData handleAttrData = new AttributeData();
121+ handleAttrData.add(tag, attr);
122+ for(int i = 0; i < handleAttrData.size; i++) {
123+ strBuf.append(" [");
124+ strBuf.append(handleAttrData.getattrname(i));
125+ strBuf.append("]");
126+ strBuf.append(handleAttrData.getcount(i));
127+ strBuf.append(" = ");
128+ strBuf.append(handleAttrData.getattrvalue(i));
129+ }
130+ }
131+ }
132+
133+ logger.log(loggerlevel, strBuf.toString());
134+ }
135+
136+ /**
137+ * デバック出力(メッセージ).
138+ * 引数に渡された任意のメッセージを出力する。
139+ * @param str メッセージ
140+ * @param methodname このメソッドを呼び出した親メソッド名
141+ */
142+ public static void htmlinfo(String str, String methodname) {
143+ logger.log(loggerlevel, str);
144+ }
145+
146+ public static void htmlinfo(String str) {
147+ logger.log(loggerlevel, str);
148+ }
149+
150+ /**
151+ * デバック出力(ＨＴＭＬ解析-本文).
152+ * 本文の内容を出力する。
153+ * @param data 本文(ＨＴＭＬ内の文字列)
154+ * @param methodname このメソッドを呼び出した親メソッド名
155+ */
156+ public static void htmlinfo(char[] data, String methodname) {
157+ String dat = new String(data);
158+ logger.log(loggerlevel, dat);
159+ }
160+
161+ public static void htmlinfo(char[] data) {
162+ String dat = new String(data);
163+ logger.log(loggerlevel, dat);
164+ }
165+
166+ /**
167+ * デバック出力(検索キー).
168+ * 検索キー(SearchData)の内容を出力する。
169+ * @param skey
170+ */
171+ public static void searchDatainfo(SearchData skey) {
172+
173+ StringBuilder strBuf = new StringBuilder(30);
174+ strBuf.append("SearchData KEY tag[");
175+ strBuf.append(skey.getHtmltag());
176+ strBuf.append("] ID[");
177+ strBuf.append(skey.getHtmlid());
178+ strBuf.append("] CLASS[");
179+ strBuf.append(skey.getHtmlclass());
180+ strBuf.append("]\n");
181+
182+ logger.log(loggerlevel, strBuf.toString());
183+ }
184+
185+ /**
186+ * ログ出力設定ファイルチェック.
187+ * 設定ファイルの存在をチェックし存在する場合、設定ファイルの内容を設定する。
188+ */
189+ private static void initLogConfiguration() {
190+
191+ File file = new File(configurationFilename);
192+ try {
193+ if(file.exists()) {
194+ FileInputStream inputStream = new FileInputStream(file);
195+ // 設定ファイルの読み込み
196+ LogManager.getLogManager().readConfiguration(inputStream);
197+ }
198+
199+ } catch (FileNotFoundException ex) {
200+ Logger.getLogger(DebugProcess.class.getName()).log(Level.SEVERE, null, ex);
201+ } catch (IOException ex) {
202+ Logger.getLogger(DebugProcess.class.getName()).log(Level.SEVERE, null, ex);
203+ }
204+ }
205+
206+ /**
207+ * ログ出力フォーマッター設定.
208+ * ファイルへログ出力時の書式を設定する。
209+ */
210+ private static void setFomatter() {
211+ Handler[] handlers = logger.getHandlers();
212+ for(int i = 0 ; i < handlers.length ; i++) {
213+ if(handlers[i] instanceof java.util.logging.FileHandler) {
214+ handlers[i].setFormatter(new HtmlFormatter());
215+ }
216+ }
217+ }
218+
219+}
220+
221+/**
222+ * ログ出力フォーマッター.
223+ * @author kgto
224+ */
225+class HtmlFormatter extends Formatter {
226+ /**
227+ * Logの出力文字列を生成する。
228+ * 出力書式：<br>
229+ * YYYY-MM-DD HH:SS:MM ログレベル<メソッド名>メッセージ
230+ */
231+ @Override
232+ public synchronized String format(final LogRecord aRecord) {
233+
234+ final StringBuffer message = new StringBuffer(100);
235+
236+ long millis = aRecord.getMillis();
237+ String time = String.format("%tF %<tT", millis);
238+
239+ message.append(time);
240+ message.append(' ');
241+
242+ message.append(aRecord.getLevel());
243+ message.append('<');
244+ String methodName = aRecord.getSourceMethodName();
245+ message.append(methodName != null ? methodName : "N/A");
246+ message.append('>');
247+
248+ message.append(formatMessage(aRecord));
249+ message.append('\n');
250+
251+ // 例外エラーの場合、エラー内容とスタックトレース出力
252+ Throwable throwable = aRecord.getThrown();
253+ if (throwable != null) {
254+ message.append(throwable.toString());
255+ message.append('\n');
256+ for (StackTraceElement trace : throwable.getStackTrace()) {
257+ message.append('\t');
258+ message.append(trace.toString());
259+ message.append('\n');
260+ }
261+ }
262+ return message.toString();
263+ }
264+}
Added: svn:keywords
## -0,0 +1 ##
+Id
\ No newline at end of property
--- branches/b4/webScraping/nbproject/project.xml (nonexistent)
+++ branches/b4/webScraping/nbproject/project.xml (revision 134)
@@ -0,0 +1,15 @@
1+<?xml version="1.0" encoding="UTF-8"?>
2+<project xmlns="http://www.netbeans.org/ns/project/1">
3+ <type>org.netbeans.modules.java.j2seproject</type>
4+ <configuration>
5+ <data xmlns="http://www.netbeans.org/ns/j2se-project/3">
6+ <name>WebScraping</name>
7+ <source-roots>
8+ <root id="src.dir"/>
9+ </source-roots>
10+ <test-roots>
11+ <root id="test.src.dir"/>
12+ </test-roots>
13+ </data>
14+ </configuration>
15+</project>
--- branches/b4/webScraping/nbproject/build-impl.xml (nonexistent)
+++ branches/b4/webScraping/nbproject/build-impl.xml (revision 134)
@@ -0,0 +1,1413 @@
1+<?xml version="1.0" encoding="UTF-8"?>
2+<!--
3+*** GENERATED FROM project.xml - DO NOT EDIT ***
4+*** EDIT ../build.xml INSTEAD ***
5+
6+For the purpose of easier reading the script
7+is divided into following sections:
8+
9+ - initialization
10+ - compilation
11+ - jar
12+ - execution
13+ - debugging
14+ - javadoc
15+ - test compilation
16+ - test execution
17+ - test debugging
18+ - applet
19+ - cleanup
20+
21+ -->
22+<project xmlns:j2seproject1="http://www.netbeans.org/ns/j2se-project/1" xmlns:j2seproject3="http://www.netbeans.org/ns/j2se-project/3" xmlns:jaxrpc="http://www.netbeans.org/ns/j2se-project/jax-rpc" basedir=".." default="default" name="WebScraping-impl">
23+ <fail message="Please build using Ant 1.8.0 or higher.">
24+ <condition>
25+ <not>
26+ <antversion atleast="1.8.0"/>
27+ </not>
28+ </condition>
29+ </fail>
30+ <target depends="test,jar,javadoc" description="Build and test whole project." name="default"/>
31+ <!--
32+ ======================
33+ INITIALIZATION SECTION
34+ ======================
35+ -->
36+ <target name="-pre-init">
37+ <!-- Empty placeholder for easier customization. -->
38+ <!-- You can override this target in the ../build.xml file. -->
39+ </target>
40+ <target depends="-pre-init" name="-init-private">
41+ <property file="nbproject/private/config.properties"/>
42+ <property file="nbproject/private/configs/${config}.properties"/>
43+ <property file="nbproject/private/private.properties"/>
44+ </target>
45+ <target depends="-pre-init,-init-private" name="-init-user">
46+ <property file="${user.properties.file}"/>
47+ <!-- The two properties below are usually overridden -->
48+ <!-- by the active platform. Just a fallback. -->
49+ <property name="default.javac.source" value="1.4"/>
50+ <property name="default.javac.target" value="1.4"/>
51+ </target>
52+ <target depends="-pre-init,-init-private,-init-user" name="-init-project">
53+ <property file="nbproject/configs/${config}.properties"/>
54+ <property file="nbproject/project.properties"/>
55+ </target>
56+ <target depends="-pre-init,-init-private,-init-user,-init-project,-init-macrodef-property" name="-do-init">
57+ <property name="platform.java" value="${java.home}/bin/java"/>
58+ <available file="${manifest.file}" property="manifest.available"/>
59+ <condition property="splashscreen.available">
60+ <and>
61+ <not>
62+ <equals arg1="${application.splash}" arg2="" trim="true"/>
63+ </not>
64+ <available file="${application.splash}"/>
65+ </and>
66+ </condition>
67+ <condition property="main.class.available">
68+ <and>
69+ <isset property="main.class"/>
70+ <not>
71+ <equals arg1="${main.class}" arg2="" trim="true"/>
72+ </not>
73+ </and>
74+ </condition>
75+ <condition property="profile.available">
76+ <and>
77+ <isset property="javac.profile"/>
78+ <length length="0" string="${javac.profile}" when="greater"/>
79+ <matches pattern="1\.[89](\..*)?" string="${javac.source}"/>
80+ </and>
81+ </condition>
82+ <condition property="do.archive">
83+ <or>
84+ <not>
85+ <istrue value="${jar.archive.disabled}"/>
86+ </not>
87+ <istrue value="${not.archive.disabled}"/>
88+ </or>
89+ </condition>
90+ <condition property="do.mkdist">
91+ <and>
92+ <isset property="do.archive"/>
93+ <isset property="libs.CopyLibs.classpath"/>
94+ <not>
95+ <istrue value="${mkdist.disabled}"/>
96+ </not>
97+ </and>
98+ </condition>
99+ <condition property="do.archive+manifest.available">
100+ <and>
101+ <isset property="manifest.available"/>
102+ <istrue value="${do.archive}"/>
103+ </and>
104+ </condition>
105+ <condition property="do.archive+main.class.available">
106+ <and>
107+ <isset property="main.class.available"/>
108+ <istrue value="${do.archive}"/>
109+ </and>
110+ </condition>
111+ <condition property="do.archive+splashscreen.available">
112+ <and>
113+ <isset property="splashscreen.available"/>
114+ <istrue value="${do.archive}"/>
115+ </and>
116+ </condition>
117+ <condition property="do.archive+profile.available">
118+ <and>
119+ <isset property="profile.available"/>
120+ <istrue value="${do.archive}"/>
121+ </and>
122+ </condition>
123+ <condition property="have.tests">
124+ <or>
125+ <available file="${test.src.dir}"/>
126+ </or>
127+ </condition>
128+ <condition property="have.sources">
129+ <or>
130+ <available file="${src.dir}"/>
131+ </or>
132+ </condition>
133+ <condition property="netbeans.home+have.tests">
134+ <and>
135+ <isset property="netbeans.home"/>
136+ <isset property="have.tests"/>
137+ </and>
138+ </condition>
139+ <condition property="no.javadoc.preview">
140+ <and>
141+ <isset property="javadoc.preview"/>
142+ <isfalse value="${javadoc.preview}"/>
143+ </and>
144+ </condition>
145+ <property name="run.jvmargs" value=""/>
146+ <property name="run.jvmargs.ide" value=""/>
147+ <property name="javac.compilerargs" value=""/>
148+ <property name="work.dir" value="${basedir}"/>
149+ <condition property="no.deps">
150+ <and>
151+ <istrue value="${no.dependencies}"/>
152+ </and>
153+ </condition>
154+ <property name="javac.debug" value="true"/>
155+ <property name="javadoc.preview" value="true"/>
156+ <property name="application.args" value=""/>
157+ <property name="source.encoding" value="${file.encoding}"/>
158+ <property name="runtime.encoding" value="${source.encoding}"/>
159+ <condition property="javadoc.encoding.used" value="${javadoc.encoding}">
160+ <and>
161+ <isset property="javadoc.encoding"/>
162+ <not>
163+ <equals arg1="${javadoc.encoding}" arg2=""/>
164+ </not>
165+ </and>
166+ </condition>
167+ <property name="javadoc.encoding.used" value="${source.encoding}"/>
168+ <property name="includes" value="**"/>
169+ <property name="excludes" value=""/>
170+ <property name="do.depend" value="false"/>
171+ <condition property="do.depend.true">
172+ <istrue value="${do.depend}"/>
173+ </condition>
174+ <path id="endorsed.classpath.path" path="${endorsed.classpath}"/>
175+ <condition else="" property="endorsed.classpath.cmd.line.arg" value="-Xbootclasspath/p:'${toString:endorsed.classpath.path}'">
176+ <and>
177+ <isset property="endorsed.classpath"/>
178+ <not>
179+ <equals arg1="${endorsed.classpath}" arg2="" trim="true"/>
180+ </not>
181+ </and>
182+ </condition>
183+ <condition else="" property="javac.profile.cmd.line.arg" value="-profile ${javac.profile}">
184+ <isset property="profile.available"/>
185+ </condition>
186+ <condition else="false" property="jdkBug6558476">
187+ <and>
188+ <matches pattern="1\.[56]" string="${java.specification.version}"/>
189+ <not>
190+ <os family="unix"/>
191+ </not>
192+ </and>
193+ </condition>
194+ <property name="javac.fork" value="${jdkBug6558476}"/>
195+ <property name="jar.index" value="false"/>
196+ <property name="jar.index.metainf" value="${jar.index}"/>
197+ <property name="copylibs.rebase" value="true"/>
198+ <available file="${meta.inf.dir}/persistence.xml" property="has.persistence.xml"/>
199+ <condition property="junit.available">
200+ <or>
201+ <available classname="org.junit.Test" classpath="${run.test.classpath}"/>
202+ <available classname="junit.framework.Test" classpath="${run.test.classpath}"/>
203+ </or>
204+ </condition>
205+ <condition property="testng.available">
206+ <available classname="org.testng.annotations.Test" classpath="${run.test.classpath}"/>
207+ </condition>
208+ <condition property="junit+testng.available">
209+ <and>
210+ <istrue value="${junit.available}"/>
211+ <istrue value="${testng.available}"/>
212+ </and>
213+ </condition>
214+ <condition else="testng" property="testng.mode" value="mixed">
215+ <istrue value="${junit+testng.available}"/>
216+ </condition>
217+ <condition else="" property="testng.debug.mode" value="-mixed">
218+ <istrue value="${junit+testng.available}"/>
219+ </condition>
220+ </target>
221+ <target name="-post-init">
222+ <!-- Empty placeholder for easier customization. -->
223+ <!-- You can override this target in the ../build.xml file. -->
224+ </target>
225+ <target depends="-pre-init,-init-private,-init-user,-init-project,-do-init" name="-init-check">
226+ <fail unless="src.dir">Must set src.dir</fail>
227+ <fail unless="test.src.dir">Must set test.src.dir</fail>
228+ <fail unless="build.dir">Must set build.dir</fail>
229+ <fail unless="dist.dir">Must set dist.dir</fail>
230+ <fail unless="build.classes.dir">Must set build.classes.dir</fail>
231+ <fail unless="dist.javadoc.dir">Must set dist.javadoc.dir</fail>
232+ <fail unless="build.test.classes.dir">Must set build.test.classes.dir</fail>
233+ <fail unless="build.test.results.dir">Must set build.test.results.dir</fail>
234+ <fail unless="build.classes.excludes">Must set build.classes.excludes</fail>
235+ <fail unless="dist.jar">Must set dist.jar</fail>
236+ </target>
237+ <target name="-init-macrodef-property">
238+ <macrodef name="property" uri="http://www.netbeans.org/ns/j2se-project/1">
239+ <attribute name="name"/>
240+ <attribute name="value"/>
241+ <sequential>
242+ <property name="@{name}" value="${@{value}}"/>
243+ </sequential>
244+ </macrodef>
245+ </target>
246+ <target depends="-init-ap-cmdline-properties" if="ap.supported.internal" name="-init-macrodef-javac-with-processors">
247+ <macrodef name="javac" uri="http://www.netbeans.org/ns/j2se-project/3">
248+ <attribute default="${src.dir}" name="srcdir"/>
249+ <attribute default="${build.classes.dir}" name="destdir"/>
250+ <attribute default="${javac.classpath}" name="classpath"/>
251+ <attribute default="${javac.processorpath}" name="processorpath"/>
252+ <attribute default="${build.generated.sources.dir}/ap-source-output" name="apgeneratedsrcdir"/>
253+ <attribute default="${includes}" name="includes"/>
254+ <attribute default="${excludes}" name="excludes"/>
255+ <attribute default="${javac.debug}" name="debug"/>
256+ <attribute default="${empty.dir}" name="sourcepath"/>
257+ <attribute default="${empty.dir}" name="gensrcdir"/>
258+ <element name="customize" optional="true"/>
259+ <sequential>
260+ <property location="${build.dir}/empty" name="empty.dir"/>
261+ <mkdir dir="${empty.dir}"/>
262+ <mkdir dir="@{apgeneratedsrcdir}"/>
263+ <javac debug="@{debug}" deprecation="${javac.deprecation}" destdir="@{destdir}" encoding="${source.encoding}" excludes="@{excludes}" fork="${javac.fork}" includeantruntime="false" includes="@{includes}" source="${javac.source}" sourcepath="@{sourcepath}" srcdir="@{srcdir}" target="${javac.target}" tempdir="${java.io.tmpdir}">
264+ <src>
265+ <dirset dir="@{gensrcdir}" erroronmissingdir="false">
266+ <include name="*"/>
267+ </dirset>
268+ </src>
269+ <classpath>
270+ <path path="@{classpath}"/>
271+ </classpath>
272+ <compilerarg line="${endorsed.classpath.cmd.line.arg}"/>
273+ <compilerarg line="${javac.profile.cmd.line.arg}"/>
274+ <compilerarg line="${javac.compilerargs}"/>
275+ <compilerarg value="-processorpath"/>
276+ <compilerarg path="@{processorpath}:${empty.dir}"/>
277+ <compilerarg line="${ap.processors.internal}"/>
278+ <compilerarg line="${annotation.processing.processor.options}"/>
279+ <compilerarg value="-s"/>
280+ <compilerarg path="@{apgeneratedsrcdir}"/>
281+ <compilerarg line="${ap.proc.none.internal}"/>
282+ <customize/>
283+ </javac>
284+ </sequential>
285+ </macrodef>
286+ </target>
287+ <target depends="-init-ap-cmdline-properties" name="-init-macrodef-javac-without-processors" unless="ap.supported.internal">
288+ <macrodef name="javac" uri="http://www.netbeans.org/ns/j2se-project/3">
289+ <attribute default="${src.dir}" name="srcdir"/>
290+ <attribute default="${build.classes.dir}" name="destdir"/>
291+ <attribute default="${javac.classpath}" name="classpath"/>
292+ <attribute default="${javac.processorpath}" name="processorpath"/>
293+ <attribute default="${build.generated.sources.dir}/ap-source-output" name="apgeneratedsrcdir"/>
294+ <attribute default="${includes}" name="includes"/>
295+ <attribute default="${excludes}" name="excludes"/>
296+ <attribute default="${javac.debug}" name="debug"/>
297+ <attribute default="${empty.dir}" name="sourcepath"/>
298+ <attribute default="${empty.dir}" name="gensrcdir"/>
299+ <element name="customize" optional="true"/>
300+ <sequential>
301+ <property location="${build.dir}/empty" name="empty.dir"/>
302+ <mkdir dir="${empty.dir}"/>
303+ <javac debug="@{debug}" deprecation="${javac.deprecation}" destdir="@{destdir}" encoding="${source.encoding}" excludes="@{excludes}" fork="${javac.fork}" includeantruntime="false" includes="@{includes}" source="${javac.source}" sourcepath="@{sourcepath}" srcdir="@{srcdir}" target="${javac.target}" tempdir="${java.io.tmpdir}">
304+ <src>
305+ <dirset dir="@{gensrcdir}" erroronmissingdir="false">
306+ <include name="*"/>
307+ </dirset>
308+ </src>
309+ <classpath>
310+ <path path="@{classpath}"/>
311+ </classpath>
312+ <compilerarg line="${endorsed.classpath.cmd.line.arg}"/>
313+ <compilerarg line="${javac.profile.cmd.line.arg}"/>
314+ <compilerarg line="${javac.compilerargs}"/>
315+ <customize/>
316+ </javac>
317+ </sequential>
318+ </macrodef>
319+ </target>
320+ <target depends="-init-macrodef-javac-with-processors,-init-macrodef-javac-without-processors" name="-init-macrodef-javac">
321+ <macrodef name="depend" uri="http://www.netbeans.org/ns/j2se-project/3">
322+ <attribute default="${src.dir}" name="srcdir"/>
323+ <attribute default="${build.classes.dir}" name="destdir"/>
324+ <attribute default="${javac.classpath}" name="classpath"/>
325+ <sequential>
326+ <depend cache="${build.dir}/depcache" destdir="@{destdir}" excludes="${excludes}" includes="${includes}" srcdir="@{srcdir}">
327+ <classpath>
328+ <path path="@{classpath}"/>
329+ </classpath>
330+ </depend>
331+ </sequential>
332+ </macrodef>
333+ <macrodef name="force-recompile" uri="http://www.netbeans.org/ns/j2se-project/3">
334+ <attribute default="${build.classes.dir}" name="destdir"/>
335+ <sequential>
336+ <fail unless="javac.includes">Must set javac.includes</fail>
337+ <pathconvert pathsep="${line.separator}" property="javac.includes.binary">
338+ <path>
339+ <filelist dir="@{destdir}" files="${javac.includes}"/>
340+ </path>
341+ <globmapper from="*.java" to="*.class"/>
342+ </pathconvert>
343+ <tempfile deleteonexit="true" property="javac.includesfile.binary"/>
344+ <echo file="${javac.includesfile.binary}" message="${javac.includes.binary}"/>
345+ <delete>
346+ <files includesfile="${javac.includesfile.binary}"/>
347+ </delete>
348+ <delete>
349+ <fileset file="${javac.includesfile.binary}"/>
350+ </delete>
351+ </sequential>
352+ </macrodef>
353+ </target>
354+ <target if="${junit.available}" name="-init-macrodef-junit-init">
355+ <condition else="false" property="nb.junit.batch" value="true">
356+ <and>
357+ <istrue value="${junit.available}"/>
358+ <not>
359+ <isset property="test.method"/>
360+ </not>
361+ </and>
362+ </condition>
363+ <condition else="false" property="nb.junit.single" value="true">
364+ <and>
365+ <istrue value="${junit.available}"/>
366+ <isset property="test.method"/>
367+ </and>
368+ </condition>
369+ </target>
370+ <target name="-init-test-properties">
371+ <property name="test.binaryincludes" value="&lt;nothing&gt;"/>
372+ <property name="test.binarytestincludes" value=""/>
373+ <property name="test.binaryexcludes" value=""/>
374+ </target>
375+ <target if="${nb.junit.single}" name="-init-macrodef-junit-single" unless="${nb.junit.batch}">
376+ <macrodef name="junit" uri="http://www.netbeans.org/ns/j2se-project/3">
377+ <attribute default="${includes}" name="includes"/>
378+ <attribute default="${excludes}" name="excludes"/>
379+ <attribute default="**" name="testincludes"/>
380+ <attribute default="" name="testmethods"/>
381+ <element name="customize" optional="true"/>
382+ <sequential>
383+ <property name="junit.forkmode" value="perTest"/>
384+ <junit dir="${work.dir}" errorproperty="tests.failed" failureproperty="tests.failed" fork="true" forkmode="${junit.forkmode}" showoutput="true" tempdir="${build.dir}">
385+ <test methods="@{testmethods}" name="@{testincludes}" todir="${build.test.results.dir}"/>
386+ <syspropertyset>
387+ <propertyref prefix="test-sys-prop."/>
388+ <mapper from="test-sys-prop.*" to="*" type="glob"/>
389+ </syspropertyset>
390+ <formatter type="brief" usefile="false"/>
391+ <formatter type="xml"/>
392+ <jvmarg value="-ea"/>
393+ <customize/>
394+ </junit>
395+ </sequential>
396+ </macrodef>
397+ </target>
398+ <target depends="-init-test-properties" if="${nb.junit.batch}" name="-init-macrodef-junit-batch" unless="${nb.junit.single}">
399+ <macrodef name="junit" uri="http://www.netbeans.org/ns/j2se-project/3">
400+ <attribute default="${includes}" name="includes"/>
401+ <attribute default="${excludes}" name="excludes"/>
402+ <attribute default="**" name="testincludes"/>
403+ <attribute default="" name="testmethods"/>
404+ <element name="customize" optional="true"/>
405+ <sequential>
406+ <property name="junit.forkmode" value="perTest"/>
407+ <junit dir="${work.dir}" errorproperty="tests.failed" failureproperty="tests.failed" fork="true" forkmode="${junit.forkmode}" showoutput="true" tempdir="${build.dir}">
408+ <batchtest todir="${build.test.results.dir}">
409+ <fileset dir="${test.src.dir}" excludes="@{excludes},${excludes}" includes="@{includes}">
410+ <filename name="@{testincludes}"/>
411+ </fileset>
412+ <fileset dir="${build.test.classes.dir}" excludes="@{excludes},${excludes},${test.binaryexcludes}" includes="${test.binaryincludes}">
413+ <filename name="${test.binarytestincludes}"/>
414+ </fileset>
415+ </batchtest>
416+ <syspropertyset>
417+ <propertyref prefix="test-sys-prop."/>
418+ <mapper from="test-sys-prop.*" to="*" type="glob"/>
419+ </syspropertyset>
420+ <formatter type="brief" usefile="false"/>
421+ <formatter type="xml"/>
422+ <jvmarg value="-ea"/>
423+ <customize/>
424+ </junit>
425+ </sequential>
426+ </macrodef>
427+ </target>
428+ <target depends="-init-macrodef-junit-init,-init-macrodef-junit-single, -init-macrodef-junit-batch" if="${junit.available}" name="-init-macrodef-junit"/>
429+ <target if="${testng.available}" name="-init-macrodef-testng">
430+ <macrodef name="testng" uri="http://www.netbeans.org/ns/j2se-project/3">
431+ <attribute default="${includes}" name="includes"/>
432+ <attribute default="${excludes}" name="excludes"/>
433+ <attribute default="**" name="testincludes"/>
434+ <attribute default="" name="testmethods"/>
435+ <element name="customize" optional="true"/>
436+ <sequential>
437+ <condition else="" property="testng.methods.arg" value="@{testincludes}.@{testmethods}">
438+ <isset property="test.method"/>
439+ </condition>
440+ <union id="test.set">
441+ <fileset dir="${test.src.dir}" excludes="@{excludes},**/*.xml,${excludes}" includes="@{includes}">
442+ <filename name="@{testincludes}"/>
443+ </fileset>
444+ </union>
445+ <taskdef classname="org.testng.TestNGAntTask" classpath="${run.test.classpath}" name="testng"/>
446+ <testng classfilesetref="test.set" failureProperty="tests.failed" listeners="org.testng.reporters.VerboseReporter" methods="${testng.methods.arg}" mode="${testng.mode}" outputdir="${build.test.results.dir}" suitename="WebScraping" testname="TestNG tests" workingDir="${work.dir}">
447+ <xmlfileset dir="${build.test.classes.dir}" includes="@{testincludes}"/>
448+ <propertyset>
449+ <propertyref prefix="test-sys-prop."/>
450+ <mapper from="test-sys-prop.*" to="*" type="glob"/>
451+ </propertyset>
452+ <customize/>
453+ </testng>
454+ </sequential>
455+ </macrodef>
456+ </target>
457+ <target name="-init-macrodef-test-impl">
458+ <macrodef name="test-impl" uri="http://www.netbeans.org/ns/j2se-project/3">
459+ <attribute default="${includes}" name="includes"/>
460+ <attribute default="${excludes}" name="excludes"/>
461+ <attribute default="**" name="testincludes"/>
462+ <attribute default="" name="testmethods"/>
463+ <element implicit="true" name="customize" optional="true"/>
464+ <sequential>
465+ <echo>No tests executed.</echo>
466+ </sequential>
467+ </macrodef>
468+ </target>
469+ <target depends="-init-macrodef-junit" if="${junit.available}" name="-init-macrodef-junit-impl">
470+ <macrodef name="test-impl" uri="http://www.netbeans.org/ns/j2se-project/3">
471+ <attribute default="${includes}" name="includes"/>
472+ <attribute default="${excludes}" name="excludes"/>
473+ <attribute default="**" name="testincludes"/>
474+ <attribute default="" name="testmethods"/>
475+ <element implicit="true" name="customize" optional="true"/>
476+ <sequential>
477+ <j2seproject3:junit excludes="@{excludes}" includes="@{includes}" testincludes="@{testincludes}" testmethods="@{testmethods}">
478+ <customize/>
479+ </j2seproject3:junit>
480+ </sequential>
481+ </macrodef>
482+ </target>
483+ <target depends="-init-macrodef-testng" if="${testng.available}" name="-init-macrodef-testng-impl">
484+ <macrodef name="test-impl" uri="http://www.netbeans.org/ns/j2se-project/3">
485+ <attribute default="${includes}" name="includes"/>
486+ <attribute default="${excludes}" name="excludes"/>
487+ <attribute default="**" name="testincludes"/>
488+ <attribute default="" name="testmethods"/>
489+ <element implicit="true" name="customize" optional="true"/>
490+ <sequential>
491+ <j2seproject3:testng excludes="@{excludes}" includes="@{includes}" testincludes="@{testincludes}" testmethods="@{testmethods}">
492+ <customize/>
493+ </j2seproject3:testng>
494+ </sequential>
495+ </macrodef>
496+ </target>
497+ <target depends="-init-macrodef-test-impl,-init-macrodef-junit-impl,-init-macrodef-testng-impl" name="-init-macrodef-test">
498+ <macrodef name="test" uri="http://www.netbeans.org/ns/j2se-project/3">
499+ <attribute default="${includes}" name="includes"/>
500+ <attribute default="${excludes}" name="excludes"/>
501+ <attribute default="**" name="testincludes"/>
502+ <attribute default="" name="testmethods"/>
503+ <sequential>
504+ <j2seproject3:test-impl excludes="@{excludes}" includes="@{includes}" testincludes="@{testincludes}" testmethods="@{testmethods}">
505+ <customize>
506+ <classpath>
507+ <path path="${run.test.classpath}"/>
508+ </classpath>
509+ <jvmarg line="${endorsed.classpath.cmd.line.arg}"/>
510+ <jvmarg line="${run.jvmargs}"/>
511+ <jvmarg line="${run.jvmargs.ide}"/>
512+ </customize>
513+ </j2seproject3:test-impl>
514+ </sequential>
515+ </macrodef>
516+ </target>
517+ <target if="${junit.available}" name="-init-macrodef-junit-debug" unless="${nb.junit.batch}">
518+ <macrodef name="junit-debug" uri="http://www.netbeans.org/ns/j2se-project/3">
519+ <attribute default="${includes}" name="includes"/>
520+ <attribute default="${excludes}" name="excludes"/>
521+ <attribute default="**" name="testincludes"/>
522+ <attribute default="" name="testmethods"/>
523+ <element name="customize" optional="true"/>
524+ <sequential>
525+ <property name="junit.forkmode" value="perTest"/>
526+ <junit dir="${work.dir}" errorproperty="tests.failed" failureproperty="tests.failed" fork="true" forkmode="${junit.forkmode}" showoutput="true" tempdir="${build.dir}">
527+ <test methods="@{testmethods}" name="@{testincludes}" todir="${build.test.results.dir}"/>
528+ <syspropertyset>
529+ <propertyref prefix="test-sys-prop."/>
530+ <mapper from="test-sys-prop.*" to="*" type="glob"/>
531+ </syspropertyset>
532+ <formatter type="brief" usefile="false"/>
533+ <formatter type="xml"/>
534+ <jvmarg value="-ea"/>
535+ <jvmarg line="${debug-args-line}"/>
536+ <jvmarg value="-Xrunjdwp:transport=${debug-transport},address=${jpda.address}"/>
537+ <customize/>
538+ </junit>
539+ </sequential>
540+ </macrodef>
541+ </target>
542+ <target depends="-init-test-properties" if="${nb.junit.batch}" name="-init-macrodef-junit-debug-batch">
543+ <macrodef name="junit-debug" uri="http://www.netbeans.org/ns/j2se-project/3">
544+ <attribute default="${includes}" name="includes"/>
545+ <attribute default="${excludes}" name="excludes"/>
546+ <attribute default="**" name="testincludes"/>
547+ <attribute default="" name="testmethods"/>
548+ <element name="customize" optional="true"/>
549+ <sequential>
550+ <property name="junit.forkmode" value="perTest"/>
551+ <junit dir="${work.dir}" errorproperty="tests.failed" failureproperty="tests.failed" fork="true" forkmode="${junit.forkmode}" showoutput="true" tempdir="${build.dir}">
552+ <batchtest todir="${build.test.results.dir}">
553+ <fileset dir="${test.src.dir}" excludes="@{excludes},${excludes}" includes="@{includes}">
554+ <filename name="@{testincludes}"/>
555+ </fileset>
556+ <fileset dir="${build.test.classes.dir}" excludes="@{excludes},${excludes},${test.binaryexcludes}" includes="${test.binaryincludes}">
557+ <filename name="${test.binarytestincludes}"/>
558+ </fileset>
559+ </batchtest>
560+ <syspropertyset>
561+ <propertyref prefix="test-sys-prop."/>
562+ <mapper from="test-sys-prop.*" to="*" type="glob"/>
563+ </syspropertyset>
564+ <formatter type="brief" usefile="false"/>
565+ <formatter type="xml"/>
566+ <jvmarg value="-ea"/>
567+ <jvmarg line="${debug-args-line}"/>
568+ <jvmarg value="-Xrunjdwp:transport=${debug-transport},address=${jpda.address}"/>
569+ <customize/>
570+ </junit>
571+ </sequential>
572+ </macrodef>
573+ </target>
574+ <target depends="-init-macrodef-junit-debug,-init-macrodef-junit-debug-batch" if="${junit.available}" name="-init-macrodef-junit-debug-impl">
575+ <macrodef name="test-debug-impl" uri="http://www.netbeans.org/ns/j2se-project/3">
576+ <attribute default="${includes}" name="includes"/>
577+ <attribute default="${excludes}" name="excludes"/>
578+ <attribute default="**" name="testincludes"/>
579+ <attribute default="" name="testmethods"/>
580+ <element implicit="true" name="customize" optional="true"/>
581+ <sequential>
582+ <j2seproject3:junit-debug excludes="@{excludes}" includes="@{includes}" testincludes="@{testincludes}" testmethods="@{testmethods}">
583+ <customize/>
584+ </j2seproject3:junit-debug>
585+ </sequential>
586+ </macrodef>
587+ </target>
588+ <target if="${testng.available}" name="-init-macrodef-testng-debug">
589+ <macrodef name="testng-debug" uri="http://www.netbeans.org/ns/j2se-project/3">
590+ <attribute default="${main.class}" name="testClass"/>
591+ <attribute default="" name="testMethod"/>
592+ <element name="customize2" optional="true"/>
593+ <sequential>
594+ <condition else="-testclass @{testClass}" property="test.class.or.method" value="-methods @{testClass}.@{testMethod}">
595+ <isset property="test.method"/>
596+ </condition>
597+ <condition else="-suitename WebScraping -testname @{testClass} ${test.class.or.method}" property="testng.cmd.args" value="@{testClass}">
598+ <matches pattern=".*\.xml" string="@{testClass}"/>
599+ </condition>
600+ <delete dir="${build.test.results.dir}" quiet="true"/>
601+ <mkdir dir="${build.test.results.dir}"/>
602+ <j2seproject3:debug classname="org.testng.TestNG" classpath="${debug.test.classpath}">
603+ <customize>
604+ <customize2/>
605+ <jvmarg value="-ea"/>
606+ <arg line="${testng.debug.mode}"/>
607+ <arg line="-d ${build.test.results.dir}"/>
608+ <arg line="-listener org.testng.reporters.VerboseReporter"/>
609+ <arg line="${testng.cmd.args}"/>
610+ </customize>
611+ </j2seproject3:debug>
612+ </sequential>
613+ </macrodef>
614+ </target>
615+ <target depends="-init-macrodef-testng-debug" if="${testng.available}" name="-init-macrodef-testng-debug-impl">
616+ <macrodef name="testng-debug-impl" uri="http://www.netbeans.org/ns/j2se-project/3">
617+ <attribute default="${main.class}" name="testClass"/>
618+ <attribute default="" name="testMethod"/>
619+ <element implicit="true" name="customize2" optional="true"/>
620+ <sequential>
621+ <j2seproject3:testng-debug testClass="@{testClass}" testMethod="@{testMethod}">
622+ <customize2/>
623+ </j2seproject3:testng-debug>
624+ </sequential>
625+ </macrodef>
626+ </target>
627+ <target depends="-init-macrodef-junit-debug-impl" if="${junit.available}" name="-init-macrodef-test-debug-junit">
628+ <macrodef name="test-debug" uri="http://www.netbeans.org/ns/j2se-project/3">
629+ <attribute default="${includes}" name="includes"/>
630+ <attribute default="${excludes}" name="excludes"/>
631+ <attribute default="**" name="testincludes"/>
632+ <attribute default="" name="testmethods"/>
633+ <attribute default="${main.class}" name="testClass"/>
634+ <attribute default="" name="testMethod"/>
635+ <sequential>
636+ <j2seproject3:test-debug-impl excludes="@{excludes}" includes="@{includes}" testincludes="@{testincludes}" testmethods="@{testmethods}">
637+ <customize>
638+ <classpath>
639+ <path path="${run.test.classpath}"/>
640+ </classpath>
641+ <jvmarg line="${endorsed.classpath.cmd.line.arg}"/>
642+ <jvmarg line="${run.jvmargs}"/>
643+ <jvmarg line="${run.jvmargs.ide}"/>
644+ </customize>
645+ </j2seproject3:test-debug-impl>
646+ </sequential>
647+ </macrodef>
648+ </target>
649+ <target depends="-init-macrodef-testng-debug-impl" if="${testng.available}" name="-init-macrodef-test-debug-testng">
650+ <macrodef name="test-debug" uri="http://www.netbeans.org/ns/j2se-project/3">
651+ <attribute default="${includes}" name="includes"/>
652+ <attribute default="${excludes}" name="excludes"/>
653+ <attribute default="**" name="testincludes"/>
654+ <attribute default="" name="testmethods"/>
655+ <attribute default="${main.class}" name="testClass"/>
656+ <attribute default="" name="testMethod"/>
657+ <sequential>
658+ <j2seproject3:testng-debug-impl testClass="@{testClass}" testMethod="@{testMethod}">
659+ <customize2>
660+ <syspropertyset>
661+ <propertyref prefix="test-sys-prop."/>
662+ <mapper from="test-sys-prop.*" to="*" type="glob"/>
663+ </syspropertyset>
664+ </customize2>
665+ </j2seproject3:testng-debug-impl>
666+ </sequential>
667+ </macrodef>
668+ </target>
669+ <target depends="-init-macrodef-test-debug-junit,-init-macrodef-test-debug-testng" name="-init-macrodef-test-debug"/>
670+ <!--
671+ pre NB7.2 profiling section; consider it deprecated
672+ -->
673+ <target depends="-profile-pre-init, init, -profile-post-init, -profile-init-macrodef-profile, -profile-init-check" if="profiler.info.jvmargs.agent" name="profile-init"/>
674+ <target if="profiler.info.jvmargs.agent" name="-profile-pre-init">
675+ <!-- Empty placeholder for easier customization. -->
676+ <!-- You can override this target in the ../build.xml file. -->
677+ </target>
678+ <target if="profiler.info.jvmargs.agent" name="-profile-post-init">
679+ <!-- Empty placeholder for easier customization. -->
680+ <!-- You can override this target in the ../build.xml file. -->
681+ </target>
682+ <target if="profiler.info.jvmargs.agent" name="-profile-init-macrodef-profile">
683+ <macrodef name="resolve">
684+ <attribute name="name"/>
685+ <attribute name="value"/>
686+ <sequential>
687+ <property name="@{name}" value="${env.@{value}}"/>
688+ </sequential>
689+ </macrodef>
690+ <macrodef name="profile">
691+ <attribute default="${main.class}" name="classname"/>
692+ <element name="customize" optional="true"/>
693+ <sequential>
694+ <property environment="env"/>
695+ <resolve name="profiler.current.path" value="${profiler.info.pathvar}"/>
696+ <java classname="@{classname}" dir="${profiler.info.dir}" fork="true" jvm="${profiler.info.jvm}">
697+ <jvmarg line="${endorsed.classpath.cmd.line.arg}"/>
698+ <jvmarg value="${profiler.info.jvmargs.agent}"/>
699+ <jvmarg line="${profiler.info.jvmargs}"/>
700+ <env key="${profiler.info.pathvar}" path="${profiler.info.agentpath}:${profiler.current.path}"/>
701+ <arg line="${application.args}"/>
702+ <classpath>
703+ <path path="${run.classpath}"/>
704+ </classpath>
705+ <syspropertyset>
706+ <propertyref prefix="run-sys-prop."/>
707+ <mapper from="run-sys-prop.*" to="*" type="glob"/>
708+ </syspropertyset>
709+ <customize/>
710+ </java>
711+ </sequential>
712+ </macrodef>
713+ </target>
714+ <target depends="-profile-pre-init, init, -profile-post-init, -profile-init-macrodef-profile" if="profiler.info.jvmargs.agent" name="-profile-init-check">
715+ <fail unless="profiler.info.jvm">Must set JVM to use for profiling in profiler.info.jvm</fail>
716+ <fail unless="profiler.info.jvmargs.agent">Must set profiler agent JVM arguments in profiler.info.jvmargs.agent</fail>
717+ </target>
718+ <!--
719+ end of pre NB7.2 profiling section
720+ -->
721+ <target depends="-init-debug-args" name="-init-macrodef-nbjpda">
722+ <macrodef name="nbjpdastart" uri="http://www.netbeans.org/ns/j2se-project/1">
723+ <attribute default="${main.class}" name="name"/>
724+ <attribute default="${debug.classpath}" name="classpath"/>
725+ <attribute default="" name="stopclassname"/>
726+ <sequential>
727+ <nbjpdastart addressproperty="jpda.address" name="@{name}" stopclassname="@{stopclassname}" transport="${debug-transport}">
728+ <classpath>
729+ <path path="@{classpath}"/>
730+ </classpath>
731+ </nbjpdastart>
732+ </sequential>
733+ </macrodef>
734+ <macrodef name="nbjpdareload" uri="http://www.netbeans.org/ns/j2se-project/1">
735+ <attribute default="${build.classes.dir}" name="dir"/>
736+ <sequential>
737+ <nbjpdareload>
738+ <fileset dir="@{dir}" includes="${fix.classes}">
739+ <include name="${fix.includes}*.class"/>
740+ </fileset>
741+ </nbjpdareload>
742+ </sequential>
743+ </macrodef>
744+ </target>
745+ <target name="-init-debug-args">
746+ <property name="version-output" value="java version &quot;${ant.java.version}"/>
747+ <condition property="have-jdk-older-than-1.4">
748+ <or>
749+ <contains string="${version-output}" substring="java version &quot;1.0"/>
750+ <contains string="${version-output}" substring="java version &quot;1.1"/>
751+ <contains string="${version-output}" substring="java version &quot;1.2"/>
752+ <contains string="${version-output}" substring="java version &quot;1.3"/>
753+ </or>
754+ </condition>
755+ <condition else="-Xdebug" property="debug-args-line" value="-Xdebug -Xnoagent -Djava.compiler=none">
756+ <istrue value="${have-jdk-older-than-1.4}"/>
757+ </condition>
758+ <condition else="dt_socket" property="debug-transport-by-os" value="dt_shmem">
759+ <os family="windows"/>
760+ </condition>
761+ <condition else="${debug-transport-by-os}" property="debug-transport" value="${debug.transport}">
762+ <isset property="debug.transport"/>
763+ </condition>
764+ </target>
765+ <target depends="-init-debug-args" name="-init-macrodef-debug">
766+ <macrodef name="debug" uri="http://www.netbeans.org/ns/j2se-project/3">
767+ <attribute default="${main.class}" name="classname"/>
768+ <attribute default="${debug.classpath}" name="classpath"/>
769+ <element name="customize" optional="true"/>
770+ <sequential>
771+ <java classname="@{classname}" dir="${work.dir}" fork="true">
772+ <jvmarg line="${endorsed.classpath.cmd.line.arg}"/>
773+ <jvmarg line="${debug-args-line}"/>
774+ <jvmarg value="-Xrunjdwp:transport=${debug-transport},address=${jpda.address}"/>
775+ <jvmarg value="-Dfile.encoding=${runtime.encoding}"/>
776+ <redirector errorencoding="${runtime.encoding}" inputencoding="${runtime.encoding}" outputencoding="${runtime.encoding}"/>
777+ <jvmarg line="${run.jvmargs}"/>
778+ <jvmarg line="${run.jvmargs.ide}"/>
779+ <classpath>
780+ <path path="@{classpath}"/>
781+ </classpath>
782+ <syspropertyset>
783+ <propertyref prefix="run-sys-prop."/>
784+ <mapper from="run-sys-prop.*" to="*" type="glob"/>
785+ </syspropertyset>
786+ <customize/>
787+ </java>
788+ </sequential>
789+ </macrodef>
790+ </target>
791+ <target name="-init-macrodef-java">
792+ <macrodef name="java" uri="http://www.netbeans.org/ns/j2se-project/1">
793+ <attribute default="${main.class}" name="classname"/>
794+ <attribute default="${run.classpath}" name="classpath"/>
795+ <attribute default="jvm" name="jvm"/>
796+ <element name="customize" optional="true"/>
797+ <sequential>
798+ <java classname="@{classname}" dir="${work.dir}" fork="true">
799+ <jvmarg line="${endorsed.classpath.cmd.line.arg}"/>
800+ <jvmarg value="-Dfile.encoding=${runtime.encoding}"/>
801+ <redirector errorencoding="${runtime.encoding}" inputencoding="${runtime.encoding}" outputencoding="${runtime.encoding}"/>
802+ <jvmarg line="${run.jvmargs}"/>
803+ <jvmarg line="${run.jvmargs.ide}"/>
804+ <classpath>
805+ <path path="@{classpath}"/>
806+ </classpath>
807+ <syspropertyset>
808+ <propertyref prefix="run-sys-prop."/>
809+ <mapper from="run-sys-prop.*" to="*" type="glob"/>
810+ </syspropertyset>
811+ <customize/>
812+ </java>
813+ </sequential>
814+ </macrodef>
815+ </target>
816+ <target name="-init-macrodef-copylibs">
817+ <macrodef name="copylibs" uri="http://www.netbeans.org/ns/j2se-project/3">
818+ <attribute default="${manifest.file}" name="manifest"/>
819+ <element name="customize" optional="true"/>
820+ <sequential>
821+ <property location="${build.classes.dir}" name="build.classes.dir.resolved"/>
822+ <pathconvert property="run.classpath.without.build.classes.dir">
823+ <path path="${run.classpath}"/>
824+ <map from="${build.classes.dir.resolved}" to=""/>
825+ </pathconvert>
826+ <pathconvert pathsep=" " property="jar.classpath">
827+ <path path="${run.classpath.without.build.classes.dir}"/>
828+ <chainedmapper>
829+ <flattenmapper/>
830+ <filtermapper>
831+ <replacestring from=" " to="%20"/>
832+ </filtermapper>
833+ <globmapper from="*" to="lib/*"/>
834+ </chainedmapper>
835+ </pathconvert>
836+ <taskdef classname="org.netbeans.modules.java.j2seproject.copylibstask.CopyLibs" classpath="${libs.CopyLibs.classpath}" name="copylibs"/>
837+ <copylibs compress="${jar.compress}" excludeFromCopy="${copylibs.excludes}" index="${jar.index}" indexMetaInf="${jar.index.metainf}" jarfile="${dist.jar}" manifest="@{manifest}" rebase="${copylibs.rebase}" runtimeclasspath="${run.classpath.without.build.classes.dir}">
838+ <fileset dir="${build.classes.dir}" excludes="${dist.archive.excludes}"/>
839+ <manifest>
840+ <attribute name="Class-Path" value="${jar.classpath}"/>
841+ <customize/>
842+ </manifest>
843+ </copylibs>
844+ </sequential>
845+ </macrodef>
846+ </target>
847+ <target name="-init-presetdef-jar">
848+ <presetdef name="jar" uri="http://www.netbeans.org/ns/j2se-project/1">
849+ <jar compress="${jar.compress}" index="${jar.index}" jarfile="${dist.jar}">
850+ <j2seproject1:fileset dir="${build.classes.dir}" excludes="${dist.archive.excludes}"/>
851+ </jar>
852+ </presetdef>
853+ </target>
854+ <target name="-init-ap-cmdline-properties">
855+ <property name="annotation.processing.enabled" value="true"/>
856+ <property name="annotation.processing.processors.list" value=""/>
857+ <property name="annotation.processing.processor.options" value=""/>
858+ <property name="annotation.processing.run.all.processors" value="true"/>
859+ <property name="javac.processorpath" value="${javac.classpath}"/>
860+ <property name="javac.test.processorpath" value="${javac.test.classpath}"/>
861+ <condition property="ap.supported.internal" value="true">
862+ <not>
863+ <matches pattern="1\.[0-5](\..*)?" string="${javac.source}"/>
864+ </not>
865+ </condition>
866+ </target>
867+ <target depends="-init-ap-cmdline-properties" if="ap.supported.internal" name="-init-ap-cmdline-supported">
868+ <condition else="" property="ap.processors.internal" value="-processor ${annotation.processing.processors.list}">
869+ <isfalse value="${annotation.processing.run.all.processors}"/>
870+ </condition>
871+ <condition else="" property="ap.proc.none.internal" value="-proc:none">
872+ <isfalse value="${annotation.processing.enabled}"/>
873+ </condition>
874+ </target>
875+ <target depends="-init-ap-cmdline-properties,-init-ap-cmdline-supported" name="-init-ap-cmdline">
876+ <property name="ap.cmd.line.internal" value=""/>
877+ </target>
878+ <target depends="-pre-init,-init-private,-init-user,-init-project,-do-init,-post-init,-init-check,-init-macrodef-property,-init-macrodef-javac,-init-macrodef-test,-init-macrodef-test-debug,-init-macrodef-nbjpda,-init-macrodef-debug,-init-macrodef-java,-init-presetdef-jar,-init-ap-cmdline" name="init"/>
879+ <!--
880+ ===================
881+ COMPILATION SECTION
882+ ===================
883+ -->
884+ <target name="-deps-jar-init" unless="built-jar.properties">
885+ <property location="${build.dir}/built-jar.properties" name="built-jar.properties"/>
886+ <delete file="${built-jar.properties}" quiet="true"/>
887+ </target>
888+ <target if="already.built.jar.${basedir}" name="-warn-already-built-jar">
889+ <echo level="warn" message="Cycle detected: WebScraping was already built"/>
890+ </target>
891+ <target depends="init,-deps-jar-init" name="deps-jar" unless="no.deps">
892+ <mkdir dir="${build.dir}"/>
893+ <touch file="${built-jar.properties}" verbose="false"/>
894+ <property file="${built-jar.properties}" prefix="already.built.jar."/>
895+ <antcall target="-warn-already-built-jar"/>
896+ <propertyfile file="${built-jar.properties}">
897+ <entry key="${basedir}" value=""/>
898+ </propertyfile>
899+ </target>
900+ <target depends="init,-check-automatic-build,-clean-after-automatic-build" name="-verify-automatic-build"/>
901+ <target depends="init" name="-check-automatic-build">
902+ <available file="${build.classes.dir}/.netbeans_automatic_build" property="netbeans.automatic.build"/>
903+ </target>
904+ <target depends="init" if="netbeans.automatic.build" name="-clean-after-automatic-build">
905+ <antcall target="clean"/>
906+ </target>
907+ <target depends="init,deps-jar" name="-pre-pre-compile">
908+ <mkdir dir="${build.classes.dir}"/>
909+ </target>
910+ <target name="-pre-compile">
911+ <!-- Empty placeholder for easier customization. -->
912+ <!-- You can override this target in the ../build.xml file. -->
913+ </target>
914+ <target if="do.depend.true" name="-compile-depend">
915+ <pathconvert property="build.generated.subdirs">
916+ <dirset dir="${build.generated.sources.dir}" erroronmissingdir="false">
917+ <include name="*"/>
918+ </dirset>
919+ </pathconvert>
920+ <j2seproject3:depend srcdir="${src.dir}:${build.generated.subdirs}"/>
921+ </target>
922+ <target depends="init,deps-jar,-pre-pre-compile,-pre-compile, -copy-persistence-xml,-compile-depend" if="have.sources" name="-do-compile">
923+ <j2seproject3:javac gensrcdir="${build.generated.sources.dir}"/>
924+ <copy todir="${build.classes.dir}">
925+ <fileset dir="${src.dir}" excludes="${build.classes.excludes},${excludes}" includes="${includes}"/>
926+ </copy>
927+ </target>
928+ <target if="has.persistence.xml" name="-copy-persistence-xml">
929+ <mkdir dir="${build.classes.dir}/META-INF"/>
930+ <copy todir="${build.classes.dir}/META-INF">
931+ <fileset dir="${meta.inf.dir}" includes="persistence.xml orm.xml"/>
932+ </copy>
933+ </target>
934+ <target name="-post-compile">
935+ <!-- Empty placeholder for easier customization. -->
936+ <!-- You can override this target in the ../build.xml file. -->
937+ </target>
938+ <target depends="init,deps-jar,-verify-automatic-build,-pre-pre-compile,-pre-compile,-do-compile,-post-compile" description="Compile project." name="compile"/>
939+ <target name="-pre-compile-single">
940+ <!-- Empty placeholder for easier customization. -->
941+ <!-- You can override this target in the ../build.xml file. -->
942+ </target>
943+ <target depends="init,deps-jar,-pre-pre-compile" name="-do-compile-single">
944+ <fail unless="javac.includes">Must select some files in the IDE or set javac.includes</fail>
945+ <j2seproject3:force-recompile/>
946+ <j2seproject3:javac excludes="" gensrcdir="${build.generated.sources.dir}" includes="${javac.includes}" sourcepath="${src.dir}"/>
947+ </target>
948+ <target name="-post-compile-single">
949+ <!-- Empty placeholder for easier customization. -->
950+ <!-- You can override this target in the ../build.xml file. -->
951+ </target>
952+ <target depends="init,deps-jar,-verify-automatic-build,-pre-pre-compile,-pre-compile-single,-do-compile-single,-post-compile-single" name="compile-single"/>
953+ <!--
954+ ====================
955+ JAR BUILDING SECTION
956+ ====================
957+ -->
958+ <target depends="init" name="-pre-pre-jar">
959+ <dirname file="${dist.jar}" property="dist.jar.dir"/>
960+ <mkdir dir="${dist.jar.dir}"/>
961+ </target>
962+ <target name="-pre-jar">
963+ <!-- Empty placeholder for easier customization. -->
964+ <!-- You can override this target in the ../build.xml file. -->
965+ </target>
966+ <target depends="init" if="do.archive" name="-do-jar-create-manifest" unless="manifest.available">
967+ <tempfile deleteonexit="true" destdir="${build.dir}" property="tmp.manifest.file"/>
968+ <touch file="${tmp.manifest.file}" verbose="false"/>
969+ </target>
970+ <target depends="init" if="do.archive+manifest.available" name="-do-jar-copy-manifest">
971+ <tempfile deleteonexit="true" destdir="${build.dir}" property="tmp.manifest.file"/>
972+ <copy file="${manifest.file}" tofile="${tmp.manifest.file}"/>
973+ </target>
974+ <target depends="init,-do-jar-create-manifest,-do-jar-copy-manifest" if="do.archive+main.class.available" name="-do-jar-set-mainclass">
975+ <manifest file="${tmp.manifest.file}" mode="update">
976+ <attribute name="Main-Class" value="${main.class}"/>
977+ </manifest>
978+ </target>
979+ <target depends="init,-do-jar-create-manifest,-do-jar-copy-manifest" if="do.archive+profile.available" name="-do-jar-set-profile">
980+ <manifest file="${tmp.manifest.file}" mode="update">
981+ <attribute name="Profile" value="${javac.profile}"/>
982+ </manifest>
983+ </target>
984+ <target depends="init,-do-jar-create-manifest,-do-jar-copy-manifest" if="do.archive+splashscreen.available" name="-do-jar-set-splashscreen">
985+ <basename file="${application.splash}" property="splashscreen.basename"/>
986+ <mkdir dir="${build.classes.dir}/META-INF"/>
987+ <copy failonerror="false" file="${application.splash}" todir="${build.classes.dir}/META-INF"/>
988+ <manifest file="${tmp.manifest.file}" mode="update">
989+ <attribute name="SplashScreen-Image" value="META-INF/${splashscreen.basename}"/>
990+ </manifest>
991+ </target>
992+ <target depends="init,-init-macrodef-copylibs,compile,-pre-pre-jar,-pre-jar,-do-jar-create-manifest,-do-jar-copy-manifest,-do-jar-set-mainclass,-do-jar-set-profile,-do-jar-set-splashscreen" if="do.mkdist" name="-do-jar-copylibs">
993+ <j2seproject3:copylibs manifest="${tmp.manifest.file}"/>
994+ <echo level="info">To run this application from the command line without Ant, try:</echo>
995+ <property location="${dist.jar}" name="dist.jar.resolved"/>
996+ <echo level="info">java -jar "${dist.jar.resolved}"</echo>
997+ </target>
998+ <target depends="init,compile,-pre-pre-jar,-pre-jar,-do-jar-create-manifest,-do-jar-copy-manifest,-do-jar-set-mainclass,-do-jar-set-profile,-do-jar-set-splashscreen" if="do.archive" name="-do-jar-jar" unless="do.mkdist">
999+ <j2seproject1:jar manifest="${tmp.manifest.file}"/>
1000+ <property location="${build.classes.dir}" name="build.classes.dir.resolved"/>
1001+ <property location="${dist.jar}" name="dist.jar.resolved"/>
1002+ <pathconvert property="run.classpath.with.dist.jar">
1003+ <path path="${run.classpath}"/>
1004+ <map from="${build.classes.dir.resolved}" to="${dist.jar.resolved}"/>
1005+ </pathconvert>
1006+ <condition else="" property="jar.usage.message" value="To run this application from the command line without Ant, try:${line.separator}${platform.java} -cp ${run.classpath.with.dist.jar} ${main.class}">
1007+ <isset property="main.class.available"/>
1008+ </condition>
1009+ <condition else="debug" property="jar.usage.level" value="info">
1010+ <isset property="main.class.available"/>
1011+ </condition>
1012+ <echo level="${jar.usage.level}" message="${jar.usage.message}"/>
1013+ </target>
1014+ <target depends="-do-jar-copylibs" if="do.archive" name="-do-jar-delete-manifest">
1015+ <delete>
1016+ <fileset file="${tmp.manifest.file}"/>
1017+ </delete>
1018+ </target>
1019+ <target depends="init,compile,-pre-pre-jar,-pre-jar,-do-jar-create-manifest,-do-jar-copy-manifest,-do-jar-set-mainclass,-do-jar-set-profile,-do-jar-set-splashscreen,-do-jar-jar,-do-jar-delete-manifest" name="-do-jar-without-libraries"/>
1020+ <target depends="init,compile,-pre-pre-jar,-pre-jar,-do-jar-create-manifest,-do-jar-copy-manifest,-do-jar-set-mainclass,-do-jar-set-profile,-do-jar-set-splashscreen,-do-jar-copylibs,-do-jar-delete-manifest" name="-do-jar-with-libraries"/>
1021+ <target name="-post-jar">
1022+ <!-- Empty placeholder for easier customization. -->
1023+ <!-- You can override this target in the ../build.xml file. -->
1024+ </target>
1025+ <target depends="init,compile,-pre-jar,-do-jar-without-libraries,-do-jar-with-libraries,-post-jar" name="-do-jar"/>
1026+ <target depends="init,compile,-pre-jar,-do-jar,-post-jar" description="Build JAR." name="jar"/>
1027+ <!--
1028+ =================
1029+ EXECUTION SECTION
1030+ =================
1031+ -->
1032+ <target depends="init,compile" description="Run a main class." name="run">
1033+ <j2seproject1:java>
1034+ <customize>
1035+ <arg line="${application.args}"/>
1036+ </customize>
1037+ </j2seproject1:java>
1038+ </target>
1039+ <target name="-do-not-recompile">
1040+ <property name="javac.includes.binary" value=""/>
1041+ </target>
1042+ <target depends="init,compile-single" name="run-single">
1043+ <fail unless="run.class">Must select one file in the IDE or set run.class</fail>
1044+ <j2seproject1:java classname="${run.class}"/>
1045+ </target>
1046+ <target depends="init,compile-test-single" name="run-test-with-main">
1047+ <fail unless="run.class">Must select one file in the IDE or set run.class</fail>
1048+ <j2seproject1:java classname="${run.class}" classpath="${run.test.classpath}"/>
1049+ </target>
1050+ <!--
1051+ =================
1052+ DEBUGGING SECTION
1053+ =================
1054+ -->
1055+ <target depends="init" if="netbeans.home" name="-debug-start-debugger">
1056+ <j2seproject1:nbjpdastart name="${debug.class}"/>
1057+ </target>
1058+ <target depends="init" if="netbeans.home" name="-debug-start-debugger-main-test">
1059+ <j2seproject1:nbjpdastart classpath="${debug.test.classpath}" name="${debug.class}"/>
1060+ </target>
1061+ <target depends="init,compile" name="-debug-start-debuggee">
1062+ <j2seproject3:debug>
1063+ <customize>
1064+ <arg line="${application.args}"/>
1065+ </customize>
1066+ </j2seproject3:debug>
1067+ </target>
1068+ <target depends="init,compile,-debug-start-debugger,-debug-start-debuggee" description="Debug project in IDE." if="netbeans.home" name="debug"/>
1069+ <target depends="init" if="netbeans.home" name="-debug-start-debugger-stepinto">
1070+ <j2seproject1:nbjpdastart stopclassname="${main.class}"/>
1071+ </target>
1072+ <target depends="init,compile,-debug-start-debugger-stepinto,-debug-start-debuggee" if="netbeans.home" name="debug-stepinto"/>
1073+ <target depends="init,compile-single" if="netbeans.home" name="-debug-start-debuggee-single">
1074+ <fail unless="debug.class">Must select one file in the IDE or set debug.class</fail>
1075+ <j2seproject3:debug classname="${debug.class}"/>
1076+ </target>
1077+ <target depends="init,compile-single,-debug-start-debugger,-debug-start-debuggee-single" if="netbeans.home" name="debug-single"/>
1078+ <target depends="init,compile-test-single" if="netbeans.home" name="-debug-start-debuggee-main-test">
1079+ <fail unless="debug.class">Must select one file in the IDE or set debug.class</fail>
1080+ <j2seproject3:debug classname="${debug.class}" classpath="${debug.test.classpath}"/>
1081+ </target>
1082+ <target depends="init,compile-test-single,-debug-start-debugger-main-test,-debug-start-debuggee-main-test" if="netbeans.home" name="debug-test-with-main"/>
1083+ <target depends="init" name="-pre-debug-fix">
1084+ <fail unless="fix.includes">Must set fix.includes</fail>
1085+ <property name="javac.includes" value="${fix.includes}.java"/>
1086+ </target>
1087+ <target depends="init,-pre-debug-fix,compile-single" if="netbeans.home" name="-do-debug-fix">
1088+ <j2seproject1:nbjpdareload/>
1089+ </target>
1090+ <target depends="init,-pre-debug-fix,-do-debug-fix" if="netbeans.home" name="debug-fix"/>
1091+ <!--
1092+ =================
1093+ PROFILING SECTION
1094+ =================
1095+ -->
1096+ <!--
1097+ pre NB7.2 profiler integration
1098+ -->
1099+ <target depends="profile-init,compile" description="Profile a project in the IDE." if="profiler.info.jvmargs.agent" name="-profile-pre72">
1100+ <fail unless="netbeans.home">This target only works when run from inside the NetBeans IDE.</fail>
1101+ <nbprofiledirect>
1102+ <classpath>
1103+ <path path="${run.classpath}"/>
1104+ </classpath>
1105+ </nbprofiledirect>
1106+ <profile/>
1107+ </target>
1108+ <target depends="profile-init,compile-single" description="Profile a selected class in the IDE." if="profiler.info.jvmargs.agent" name="-profile-single-pre72">
1109+ <fail unless="profile.class">Must select one file in the IDE or set profile.class</fail>
1110+ <fail unless="netbeans.home">This target only works when run from inside the NetBeans IDE.</fail>
1111+ <nbprofiledirect>
1112+ <classpath>
1113+ <path path="${run.classpath}"/>
1114+ </classpath>
1115+ </nbprofiledirect>
1116+ <profile classname="${profile.class}"/>
1117+ </target>
1118+ <target depends="profile-init,compile-single" if="profiler.info.jvmargs.agent" name="-profile-applet-pre72">
1119+ <fail unless="netbeans.home">This target only works when run from inside the NetBeans IDE.</fail>
1120+ <nbprofiledirect>
1121+ <classpath>
1122+ <path path="${run.classpath}"/>
1123+ </classpath>
1124+ </nbprofiledirect>
1125+ <profile classname="sun.applet.AppletViewer">
1126+ <customize>
1127+ <arg value="${applet.url}"/>
1128+ </customize>
1129+ </profile>
1130+ </target>
1131+ <target depends="profile-init,compile-test-single" if="profiler.info.jvmargs.agent" name="-profile-test-single-pre72">
1132+ <fail unless="netbeans.home">This target only works when run from inside the NetBeans IDE.</fail>
1133+ <nbprofiledirect>
1134+ <classpath>
1135+ <path path="${run.test.classpath}"/>
1136+ </classpath>
1137+ </nbprofiledirect>
1138+ <junit dir="${profiler.info.dir}" errorproperty="tests.failed" failureproperty="tests.failed" fork="true" jvm="${profiler.info.jvm}" showoutput="true">
1139+ <env key="${profiler.info.pathvar}" path="${profiler.info.agentpath}:${profiler.current.path}"/>
1140+ <jvmarg value="${profiler.info.jvmargs.agent}"/>
1141+ <jvmarg line="${profiler.info.jvmargs}"/>
1142+ <test name="${profile.class}"/>
1143+ <classpath>
1144+ <path path="${run.test.classpath}"/>
1145+ </classpath>
1146+ <syspropertyset>
1147+ <propertyref prefix="test-sys-prop."/>
1148+ <mapper from="test-sys-prop.*" to="*" type="glob"/>
1149+ </syspropertyset>
1150+ <formatter type="brief" usefile="false"/>
1151+ <formatter type="xml"/>
1152+ </junit>
1153+ </target>
1154+ <!--
1155+ end of pre NB72 profiling section
1156+ -->
1157+ <target if="netbeans.home" name="-profile-check">
1158+ <condition property="profiler.configured">
1159+ <or>
1160+ <contains casesensitive="true" string="${run.jvmargs.ide}" substring="-agentpath:"/>
1161+ <contains casesensitive="true" string="${run.jvmargs.ide}" substring="-javaagent:"/>
1162+ </or>
1163+ </condition>
1164+ </target>
1165+ <target depends="-profile-check,-profile-pre72" description="Profile a project in the IDE." if="profiler.configured" name="profile" unless="profiler.info.jvmargs.agent">
1166+ <startprofiler/>
1167+ <antcall target="run"/>
1168+ </target>
1169+ <target depends="-profile-check,-profile-single-pre72" description="Profile a selected class in the IDE." if="profiler.configured" name="profile-single" unless="profiler.info.jvmargs.agent">
1170+ <fail unless="run.class">Must select one file in the IDE or set run.class</fail>
1171+ <startprofiler/>
1172+ <antcall target="run-single"/>
1173+ </target>
1174+ <target depends="-profile-test-single-pre72" description="Profile a selected test in the IDE." name="profile-test-single"/>
1175+ <target depends="-profile-check" description="Profile a selected test in the IDE." if="profiler.configured" name="profile-test" unless="profiler.info.jvmargs">
1176+ <fail unless="test.includes">Must select some files in the IDE or set test.includes</fail>
1177+ <startprofiler/>
1178+ <antcall target="test-single"/>
1179+ </target>
1180+ <target depends="-profile-check" description="Profile a selected class in the IDE." if="profiler.configured" name="profile-test-with-main">
1181+ <fail unless="run.class">Must select one file in the IDE or set run.class</fail>
1182+ <startprofiler/>
1183+ <antcal target="run-test-with-main"/>
1184+ </target>
1185+ <target depends="-profile-check,-profile-applet-pre72" if="profiler.configured" name="profile-applet" unless="profiler.info.jvmargs.agent">
1186+ <fail unless="applet.url">Must select one file in the IDE or set applet.url</fail>
1187+ <startprofiler/>
1188+ <antcall target="run-applet"/>
1189+ </target>
1190+ <!--
1191+ ===============
1192+ JAVADOC SECTION
1193+ ===============
1194+ -->
1195+ <target depends="init" if="have.sources" name="-javadoc-build">
1196+ <mkdir dir="${dist.javadoc.dir}"/>
1197+ <condition else="" property="javadoc.endorsed.classpath.cmd.line.arg" value="-J${endorsed.classpath.cmd.line.arg}">
1198+ <and>
1199+ <isset property="endorsed.classpath.cmd.line.arg"/>
1200+ <not>
1201+ <equals arg1="${endorsed.classpath.cmd.line.arg}" arg2=""/>
1202+ </not>
1203+ </and>
1204+ </condition>
1205+ <condition else="" property="bug5101868workaround" value="*.java">
1206+ <matches pattern="1\.[56](\..*)?" string="${java.version}"/>
1207+ </condition>
1208+ <javadoc additionalparam="-J-Dfile.encoding=${file.encoding} ${javadoc.additionalparam}" author="${javadoc.author}" charset="UTF-8" destdir="${dist.javadoc.dir}" docencoding="UTF-8" encoding="${javadoc.encoding.used}" failonerror="true" noindex="${javadoc.noindex}" nonavbar="${javadoc.nonavbar}" notree="${javadoc.notree}" private="${javadoc.private}" source="${javac.source}" splitindex="${javadoc.splitindex}" use="${javadoc.use}" useexternalfile="true" version="${javadoc.version}" windowtitle="${javadoc.windowtitle}">
1209+ <classpath>
1210+ <path path="${javac.classpath}"/>
1211+ </classpath>
1212+ <fileset dir="${src.dir}" excludes="${bug5101868workaround},${excludes}" includes="${includes}">
1213+ <filename name="**/*.java"/>
1214+ </fileset>
1215+ <fileset dir="${build.generated.sources.dir}" erroronmissingdir="false">
1216+ <include name="**/*.java"/>
1217+ <exclude name="*.java"/>
1218+ </fileset>
1219+ <arg line="${javadoc.endorsed.classpath.cmd.line.arg}"/>
1220+ </javadoc>
1221+ <copy todir="${dist.javadoc.dir}">
1222+ <fileset dir="${src.dir}" excludes="${excludes}" includes="${includes}">
1223+ <filename name="**/doc-files/**"/>
1224+ </fileset>
1225+ <fileset dir="${build.generated.sources.dir}" erroronmissingdir="false">
1226+ <include name="**/doc-files/**"/>
1227+ </fileset>
1228+ </copy>
1229+ </target>
1230+ <target depends="init,-javadoc-build" if="netbeans.home" name="-javadoc-browse" unless="no.javadoc.preview">
1231+ <nbbrowse file="${dist.javadoc.dir}/index.html"/>
1232+ </target>
1233+ <target depends="init,-javadoc-build,-javadoc-browse" description="Build Javadoc." name="javadoc"/>
1234+ <!--
1235+ =========================
1236+ TEST COMPILATION SECTION
1237+ =========================
1238+ -->
1239+ <target depends="init,compile" if="have.tests" name="-pre-pre-compile-test">
1240+ <mkdir dir="${build.test.classes.dir}"/>
1241+ </target>
1242+ <target name="-pre-compile-test">
1243+ <!-- Empty placeholder for easier customization. -->
1244+ <!-- You can override this target in the ../build.xml file. -->
1245+ </target>
1246+ <target if="do.depend.true" name="-compile-test-depend">
1247+ <j2seproject3:depend classpath="${javac.test.classpath}" destdir="${build.test.classes.dir}" srcdir="${test.src.dir}"/>
1248+ </target>
1249+ <target depends="init,deps-jar,compile,-pre-pre-compile-test,-pre-compile-test,-compile-test-depend" if="have.tests" name="-do-compile-test">
1250+ <j2seproject3:javac apgeneratedsrcdir="${build.test.classes.dir}" classpath="${javac.test.classpath}" debug="true" destdir="${build.test.classes.dir}" processorpath="${javac.test.processorpath}" srcdir="${test.src.dir}"/>
1251+ <copy todir="${build.test.classes.dir}">
1252+ <fileset dir="${test.src.dir}" excludes="${build.classes.excludes},${excludes}" includes="${includes}"/>
1253+ </copy>
1254+ </target>
1255+ <target name="-post-compile-test">
1256+ <!-- Empty placeholder for easier customization. -->
1257+ <!-- You can override this target in the ../build.xml file. -->
1258+ </target>
1259+ <target depends="init,compile,-pre-pre-compile-test,-pre-compile-test,-do-compile-test,-post-compile-test" name="compile-test"/>
1260+ <target name="-pre-compile-test-single">
1261+ <!-- Empty placeholder for easier customization. -->
1262+ <!-- You can override this target in the ../build.xml file. -->
1263+ </target>
1264+ <target depends="init,deps-jar,compile,-pre-pre-compile-test,-pre-compile-test-single" if="have.tests" name="-do-compile-test-single">
1265+ <fail unless="javac.includes">Must select some files in the IDE or set javac.includes</fail>
1266+ <j2seproject3:force-recompile destdir="${build.test.classes.dir}"/>
1267+ <j2seproject3:javac apgeneratedsrcdir="${build.test.classes.dir}" classpath="${javac.test.classpath}" debug="true" destdir="${build.test.classes.dir}" excludes="" includes="${javac.includes}" processorpath="${javac.test.processorpath}" sourcepath="${test.src.dir}" srcdir="${test.src.dir}"/>
1268+ <copy todir="${build.test.classes.dir}">
1269+ <fileset dir="${test.src.dir}" excludes="${build.classes.excludes},${excludes}" includes="${includes}"/>
1270+ </copy>
1271+ </target>
1272+ <target name="-post-compile-test-single">
1273+ <!-- Empty placeholder for easier customization. -->
1274+ <!-- You can override this target in the ../build.xml file. -->
1275+ </target>
1276+ <target depends="init,compile,-pre-pre-compile-test,-pre-compile-test-single,-do-compile-test-single,-post-compile-test-single" name="compile-test-single"/>
1277+ <!--
1278+ =======================
1279+ TEST EXECUTION SECTION
1280+ =======================
1281+ -->
1282+ <target depends="init" if="have.tests" name="-pre-test-run">
1283+ <mkdir dir="${build.test.results.dir}"/>
1284+ </target>
1285+ <target depends="init,compile-test,-pre-test-run" if="have.tests" name="-do-test-run">
1286+ <j2seproject3:test includes="${includes}" testincludes="**/*Test.java"/>
1287+ </target>
1288+ <target depends="init,compile-test,-pre-test-run,-do-test-run" if="have.tests" name="-post-test-run">
1289+ <fail if="tests.failed" unless="ignore.failing.tests">Some tests failed; see details above.</fail>
1290+ </target>
1291+ <target depends="init" if="have.tests" name="test-report"/>
1292+ <target depends="init" if="netbeans.home+have.tests" name="-test-browse"/>
1293+ <target depends="init,compile-test,-pre-test-run,-do-test-run,test-report,-post-test-run,-test-browse" description="Run unit tests." name="test"/>
1294+ <target depends="init" if="have.tests" name="-pre-test-run-single">
1295+ <mkdir dir="${build.test.results.dir}"/>
1296+ </target>
1297+ <target depends="init,compile-test-single,-pre-test-run-single" if="have.tests" name="-do-test-run-single">
1298+ <fail unless="test.includes">Must select some files in the IDE or set test.includes</fail>
1299+ <j2seproject3:test excludes="" includes="${test.includes}" testincludes="${test.includes}"/>
1300+ </target>
1301+ <target depends="init,compile-test-single,-pre-test-run-single,-do-test-run-single" if="have.tests" name="-post-test-run-single">
1302+ <fail if="tests.failed" unless="ignore.failing.tests">Some tests failed; see details above.</fail>
1303+ </target>
1304+ <target depends="init,compile-test-single,-pre-test-run-single,-do-test-run-single,-post-test-run-single" description="Run single unit test." name="test-single"/>
1305+ <target depends="init,compile-test-single,-pre-test-run-single" if="have.tests" name="-do-test-run-single-method">
1306+ <fail unless="test.class">Must select some files in the IDE or set test.class</fail>
1307+ <fail unless="test.method">Must select some method in the IDE or set test.method</fail>
1308+ <j2seproject3:test excludes="" includes="${javac.includes}" testincludes="${test.class}" testmethods="${test.method}"/>
1309+ </target>
1310+ <target depends="init,compile-test-single,-pre-test-run-single,-do-test-run-single-method" if="have.tests" name="-post-test-run-single-method">
1311+ <fail if="tests.failed" unless="ignore.failing.tests">Some tests failed; see details above.</fail>
1312+ </target>
1313+ <target depends="init,compile-test-single,-pre-test-run-single,-do-test-run-single-method,-post-test-run-single-method" description="Run single unit test." name="test-single-method"/>
1314+ <!--
1315+ =======================
1316+ TEST DEBUGGING SECTION
1317+ =======================
1318+ -->
1319+ <target depends="init,compile-test-single,-pre-test-run-single" if="have.tests" name="-debug-start-debuggee-test">
1320+ <fail unless="test.class">Must select one file in the IDE or set test.class</fail>
1321+ <j2seproject3:test-debug excludes="" includes="${javac.includes}" testClass="${test.class}" testincludes="${javac.includes}"/>
1322+ </target>
1323+ <target depends="init,compile-test-single,-pre-test-run-single" if="have.tests" name="-debug-start-debuggee-test-method">
1324+ <fail unless="test.class">Must select one file in the IDE or set test.class</fail>
1325+ <fail unless="test.method">Must select some method in the IDE or set test.method</fail>
1326+ <j2seproject3:test-debug excludes="" includes="${javac.includes}" testClass="${test.class}" testMethod="${test.method}" testincludes="${test.class}" testmethods="${test.method}"/>
1327+ </target>
1328+ <target depends="init,compile-test" if="netbeans.home+have.tests" name="-debug-start-debugger-test">
1329+ <j2seproject1:nbjpdastart classpath="${debug.test.classpath}" name="${test.class}"/>
1330+ </target>
1331+ <target depends="init,compile-test-single,-debug-start-debugger-test,-debug-start-debuggee-test" name="debug-test"/>
1332+ <target depends="init,compile-test-single,-debug-start-debugger-test,-debug-start-debuggee-test-method" name="debug-test-method"/>
1333+ <target depends="init,-pre-debug-fix,compile-test-single" if="netbeans.home" name="-do-debug-fix-test">
1334+ <j2seproject1:nbjpdareload dir="${build.test.classes.dir}"/>
1335+ </target>
1336+ <target depends="init,-pre-debug-fix,-do-debug-fix-test" if="netbeans.home" name="debug-fix-test"/>
1337+ <!--
1338+ =========================
1339+ APPLET EXECUTION SECTION
1340+ =========================
1341+ -->
1342+ <target depends="init,compile-single" name="run-applet">
1343+ <fail unless="applet.url">Must select one file in the IDE or set applet.url</fail>
1344+ <j2seproject1:java classname="sun.applet.AppletViewer">
1345+ <customize>
1346+ <arg value="${applet.url}"/>
1347+ </customize>
1348+ </j2seproject1:java>
1349+ </target>
1350+ <!--
1351+ =========================
1352+ APPLET DEBUGGING SECTION
1353+ =========================
1354+ -->
1355+ <target depends="init,compile-single" if="netbeans.home" name="-debug-start-debuggee-applet">
1356+ <fail unless="applet.url">Must select one file in the IDE or set applet.url</fail>
1357+ <j2seproject3:debug classname="sun.applet.AppletViewer">
1358+ <customize>
1359+ <arg value="${applet.url}"/>
1360+ </customize>
1361+ </j2seproject3:debug>
1362+ </target>
1363+ <target depends="init,compile-single,-debug-start-debugger,-debug-start-debuggee-applet" if="netbeans.home" name="debug-applet"/>
1364+ <!--
1365+ ===============
1366+ CLEANUP SECTION
1367+ ===============
1368+ -->
1369+ <target name="-deps-clean-init" unless="built-clean.properties">
1370+ <property location="${build.dir}/built-clean.properties" name="built-clean.properties"/>
1371+ <delete file="${built-clean.properties}" quiet="true"/>
1372+ </target>
1373+ <target if="already.built.clean.${basedir}" name="-warn-already-built-clean">
1374+ <echo level="warn" message="Cycle detected: WebScraping was already built"/>
1375+ </target>
1376+ <target depends="init,-deps-clean-init" name="deps-clean" unless="no.deps">
1377+ <mkdir dir="${build.dir}"/>
1378+ <touch file="${built-clean.properties}" verbose="false"/>
1379+ <property file="${built-clean.properties}" prefix="already.built.clean."/>
1380+ <antcall target="-warn-already-built-clean"/>
1381+ <propertyfile file="${built-clean.properties}">
1382+ <entry key="${basedir}" value=""/>
1383+ </propertyfile>
1384+ </target>
1385+ <target depends="init" name="-do-clean">
1386+ <delete dir="${build.dir}"/>
1387+ <delete dir="${dist.dir}" followsymlinks="false" includeemptydirs="true"/>
1388+ </target>
1389+ <target name="-post-clean">
1390+ <!-- Empty placeholder for easier customization. -->
1391+ <!-- You can override this target in the ../build.xml file. -->
1392+ </target>
1393+ <target depends="init,deps-clean,-do-clean,-post-clean" description="Clean build products." name="clean"/>
1394+ <target name="-check-call-dep">
1395+ <property file="${call.built.properties}" prefix="already.built."/>
1396+ <condition property="should.call.dep">
1397+ <and>
1398+ <not>
1399+ <isset property="already.built.${call.subproject}"/>
1400+ </not>
1401+ <available file="${call.script}"/>
1402+ </and>
1403+ </condition>
1404+ </target>
1405+ <target depends="-check-call-dep" if="should.call.dep" name="-maybe-call-dep">
1406+ <ant antfile="${call.script}" inheritall="false" target="${call.target}">
1407+ <propertyset>
1408+ <propertyref prefix="transfer."/>
1409+ <mapper from="transfer.*" to="*" type="glob"/>
1410+ </propertyset>
1411+ </ant>
1412+ </target>
1413+</project>
--- branches/b4/webScraping/build.xml (nonexistent)
+++ branches/b4/webScraping/build.xml (revision 134)
@@ -0,0 +1,73 @@
1+<?xml version="1.0" encoding="UTF-8"?>
2+<!-- You may freely edit this file. See commented blocks below for -->
3+<!-- some examples of how to customize the build. -->
4+<!-- (If you delete it and reopen the project it will be recreated.) -->
5+<!-- By default, only the Clean and Build commands use this build script. -->
6+<!-- Commands such as Run, Debug, and Test only use this build script if -->
7+<!-- the Compile on Save feature is turned off for the project. -->
8+<!-- You can turn off the Compile on Save (or Deploy on Save) setting -->
9+<!-- in the project's Project Properties dialog box.-->
10+<project name="WebScraping" default="default" basedir=".">
11+ <description>Builds, tests, and runs the project WebScraping.</description>
12+ <import file="nbproject/build-impl.xml"/>
13+ <!--
14+
15+ There exist several targets which are by default empty and which can be
16+ used for execution of your tasks. These targets are usually executed
17+ before and after some main targets. They are:
18+
19+ -pre-init: called before initialization of project properties
20+ -post-init: called after initialization of project properties
21+ -pre-compile: called before javac compilation
22+ -post-compile: called after javac compilation
23+ -pre-compile-single: called before javac compilation of single file
24+ -post-compile-single: called after javac compilation of single file
25+ -pre-compile-test: called before javac compilation of JUnit tests
26+ -post-compile-test: called after javac compilation of JUnit tests
27+ -pre-compile-test-single: called before javac compilation of single JUnit test
28+ -post-compile-test-single: called after javac compilation of single JUunit test
29+ -pre-jar: called before JAR building
30+ -post-jar: called after JAR building
31+ -post-clean: called after cleaning build products
32+
33+ (Targets beginning with '-' are not intended to be called on their own.)
34+
35+ Example of inserting an obfuscator after compilation could look like this:
36+
37+ <target name="-post-compile">
38+ <obfuscate>
39+ <fileset dir="${build.classes.dir}"/>
40+ </obfuscate>
41+ </target>
42+
43+ For list of available properties check the imported
44+ nbproject/build-impl.xml file.
45+
46+
47+ Another way to customize the build is by overriding existing main targets.
48+ The targets of interest are:
49+
50+ -init-macrodef-javac: defines macro for javac compilation
51+ -init-macrodef-junit: defines macro for junit execution
52+ -init-macrodef-debug: defines macro for class debugging
53+ -init-macrodef-java: defines macro for class execution
54+ -do-jar: JAR building
55+ run: execution of project
56+ -javadoc-build: Javadoc generation
57+ test-report: JUnit report generation
58+
59+ An example of overriding the target for project execution could look like this:
60+
61+ <target name="run" depends="WebScraping-impl.jar">
62+ <exec dir="bin" executable="launcher.exe">
63+ <arg file="${dist.jar}"/>
64+ </exec>
65+ </target>
66+
67+ Notice that the overridden target depends on the jar target and not only on
68+ the compile target as the regular run target does. Again, for a list of available
69+ properties which you can use, check the target you are overriding in the
70+ nbproject/build-impl.xml file.
71+
72+ -->
73+</project>
--- branches/b4/webScraping/manifest.mf (nonexistent)
+++ branches/b4/webScraping/manifest.mf (revision 134)
@@ -0,0 +1,3 @@
1+Manifest-Version: 1.0
2+X-COMMENT: Main-Class will be added automatically by build
3+