Open in new window / Try shogun cloud
--- Log opened Sun Apr 06 00:00:10 2014
-!- diver-in-sky [uid28350@gateway/web/irccloud.com/x-okkuwpmicheahbzm] has joined #shogun00:30
-!- kislay [~kislay@117.239.94.112] has joined #shogun00:57
lisitsyn1diver-in-sky: welcome home ;)01:36
-!- soumyaC [uid15286@gateway/web/irccloud.com/x-bnghsadaioaqovax] has joined #shogun09:32
-!- kprah [67157e4c@gateway/web/freenode/ip.103.21.126.76] has joined #shogun10:09
-!- newlido [~quassel@41.218.172.126] has joined #shogun11:06
-!- kprah [67157e4c@gateway/web/freenode/ip.103.21.126.76] has quit [Quit: Page closed]11:13
-!- kislay [~kislay@117.239.94.112] has quit [Read error: Connection reset by peer]11:40
-!- kprah [67157f4c@gateway/web/freenode/ip.103.21.127.76] has joined #shogun12:00
-!- kislay [~kislay@117.239.94.112] has joined #shogun12:02
-!- kprah [67157f4c@gateway/web/freenode/ip.103.21.127.76] has quit [Quit: Page closed]12:14
-!- thoralf [~thoralf@91-65-137-212-dynip.superkabel.de] has joined #shogun12:38
thoralfHeyho.12:38
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun12:39
shogun-notifier-shogun: Thoralf Klein :develop * 2f61caf / src/shogun/base/init.cpp: https://github.com/shogun-toolbox/shogun/commit/2f61caf066674b27beb8cfca33adcd6187489a0612:39
shogun-notifier-shogun: Cleaning up protobuf library in exit_shogun().12:39
shogun-notifier-shogun: tklein23 :develop * 11c25d3 / src/shogun/base/init.cpp: https://github.com/shogun-toolbox/shogun/commit/11c25d37eaa73a4b0a9210a6d70da4685cf3bc4012:39
shogun-notifier-shogun: Merge pull request #2107 from tklein23/fix_protobuf_leak12:39
shogun-notifier-shogun:12:39
shogun-notifier-shogun: Shutting down protobuf library in exit_shogun() to fix memory leak.12:39
-!- newlido [~quassel@41.218.172.126] has quit [Remote host closed the connection]12:50
-!- newlido [~quassel@41.218.172.126] has joined #shogun12:53
-!- kprah [67157e4c@gateway/web/freenode/ip.103.21.126.76] has joined #shogun12:56
thoralfkprah: Have a look how we're reading files in the unit tests or examples.13:13
thoralfkprah: I think there are plenty of examples.13:13
thoralfDo you want your file to be part of the unit tests, i.e. you're committing this file to the repository?13:14
kprahyeah.13:14
kprahThis file was generated in pythn13:14
kprahas you see in the issue13:14
kprahmy fix works locally.13:15
thoralfDoes the created file exist in the repository already or how should it be created before running your test?13:15
kprahthoralf: generating such a matrix from c++ gives no errors.13:16
kprahthoralf: I mean it works perfectly. But while reading file generated by this way,13:16
kprahthoralf: we get an extra feature. So my fix takes care of both the cases locally.13:17
kprahthoralf: By the way, yeah it should be generated before test13:17
kprahany particular examples?13:19
kprahthoralf?13:19
thoralfWhere does the error occur?  While saving or while loading?13:29
-!- iglesiasg [~iglesiasg@82.74.224.167] has joined #shogun13:29
-!- mode/#shogun [+o iglesiasg] by ChanServ13:30
thoralfHey iglesiasg13:34
@iglesiasgthoralf, sup13:34
thoralfeverything sup ;)13:35
@iglesiasghaha that's good then :)13:36
kprahsorry.13:37
kprahWas away for a few minutes13:37
kprahError occurs while loading by ctest after make install step13:38
thoralfkprah: No, I mean the error you're working on.13:38
thoralfThe serialization error.13:38
thoralfThis error happens while saving or loading?13:38
kprahloading13:39
thoralfWell, then commit this example file to shogun-data?13:39
thoralfSo you can load it from inside the test.13:39
thoralfGood weather outside.  Need to charge my batteries in the sun.  See you!13:39
kprahok. What should be the filename then?13:40
kprahthoralf?13:40
thoralfDon't know.  test_input_for_bug_1991.csv?13:40
thoralfAway now.13:40
kprahok. Thank you very much. Bye!13:41
kprahhello, iglesiasg13:41
@iglesiasghello kprah13:41
kprahI was discussing a doubt with thoralf just now.13:42
kprahiglesiasg: can you help me?13:42
@iglesiasgkprah, let me check the logs13:42
kprahiglesiasg: most of it happened in the private chat13:43
kprahiglesiasg: can I explain?13:43
@iglesiasgkprah, sure. Go ahead please13:43
kprahiglesiasg: I am working on issue 1991.13:43
@iglesiasgkprah, ok, what have you done so far?13:44
kprahI found the bug and fixed it locally. But I am facing a strange problem while loading the test_20k.csv13:44
kprahiglesiasg: can you see it?13:44
-!- thoralf [~thoralf@91-65-137-212-dynip.superkabel.de] has quit [Ping timeout: 255 seconds]13:45
@iglesiasgkprah, I am reading the issue yes, if that's what you mean13:45
@iglesiasgkprah, I see you opened a pull request as well13:46
kprahiglesiasg: yeah.13:46
@iglesiasgkislay, although I don't think we want those csv files in the repository13:46
@iglesiasgkprah, oh sorry, I talked to the wrong person :)13:47
kislayiglesiasg, :D13:47
kprahbut the unit tests fail when ctest runs. I checked it in travis logs. The reason is it is unable to load that csv file.13:47
kprahiglesiasg: any insights on how to load this file?13:47
@iglesiasgkprah, what are the dependencies of CSVFile? Maybe there is something not installed in travis and that's why the error13:48
kprahCSVFile depends on three classes, CLineReader, CDelimiterTokenizer and CParser13:49
kprahiglesiasg: Well in the unit test I need that csv file generated by np.random.rand(20000, 70)13:49
kprahiglesiasg: The fact is when I run the unit test via ./shogun-unit-test13:50
-!- HeikoS [~heiko@2.248.35.30] has joined #shogun13:50
-!- mode/#shogun [+o HeikoS] by ChanServ13:50
kprahit runs fine13:50
@iglesiasgkprah, I meant dependencies as installed libraries, not other classes13:50
kprahIt is a core class of shogun only13:51
kprahdoesn't depend on other libraries13:51
@iglesiasgkprah, all right13:51
@iglesiasgkprah, have you found out what the exact line where the error arises is13:52
kprahiglesiasg:13:53
kprahyeah13:53
kprahhttp://pastebin.com/1w20Ubpg13:53
kprahiglesiasg: here while loading test_20k.csv, ctest fails.13:54
kprahprobably because ctest runs from a different directory13:54
@iglesiasgkprah, could it be related to the path to the file?13:54
kprahYeah,13:55
kprahI thought of using absolute pathnames but that is not a good practice.13:55
@iglesiasgkprah, why don't you just generate random data within the test?13:55
kprahiglesiasg: That works fine. There is a unit test already present. See CSVFile_unittest.cc -> matrix_int3213:56
kprahiglesiasg: The bug was found days ago by one of the guests who reported it on IRC13:57
@iglesiasgkprah, I meant to generate random data in the test, save it into a temporary file, do the test and then remove the file13:57
kprahiglesiasg: Yeah. There is a test for that already. It is the test that I just mentioned above13:58
kprahand it passes well.13:59
kprahit only doesn't pass for the ones generated from numpy , as described in the issue by viktor (wiking)13:59
kprahso I thought maybe loading the file directly would be fine14:00
@iglesiasgkprah, I don't think the problem is related to creating the file from numpy..14:01
kprahiglesiasg: Well I used the file given by this, and confirmed that there is indeed an extra feature.14:03
kprahiglesiasg: But if you do it by the one generated from libshogun14:03
kprahit just works fine.14:04
kprahCan I post the screenshots?14:04
kprahWell actually ..... You are right. THe bug was in CSVFile class itself14:05
@iglesiasgkprah, why don't you just try to see if the problem is because of the path to the file?14:05
kprahwhere, in the unit test?14:05
kprahiglesiasg?14:06
@iglesiasgkprah, run locally the tests exactly as travis does, from the same directory, using the same commands. You should be able to see this in the travis logs14:06
kprahYeah. I did it locally too.14:06
kprahit failed while loading the file.14:06
kprahbut when I make a separate program14:06
kprahusing int main() and do all my things there it works fine (as it should)14:06
kprahiglesiasg?14:07
@iglesiasgkprah, I cannot tell you more apart from what I already said :)14:08
kprahSo the main difficulty that I am facing right now is while loading this file in my unit test.I fixed the bug otherwise, and confirmed it by making a separate program.14:08
kprahiglesiasg?14:08
-!- HeikoS [~heiko@2.248.35.30] has quit [Quit: Leaving.]14:30
-!- iglesiasg [~iglesiasg@82.74.224.167] has quit [Quit: Leaving]14:35
-!- kprah [67157e4c@gateway/web/freenode/ip.103.21.126.76] has left #shogun []14:57
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]15:39
-!- kislay [~kislay@117.239.94.112] has quit [Read error: Connection reset by peer]16:24
-!- sonne|work [~sonnenbu@91-64-78-194-dynip.superkabel.de] has quit [Ping timeout: 255 seconds]16:27
-!- sonne|work [~sonnenbu@sams-office-nat.tomtomgroup.com] has joined #shogun16:28
-!- HeikoS [~heiko@90-227-238-158-no15.business.telia.com] has joined #shogun16:46
-!- mode/#shogun [+o HeikoS] by ChanServ16:46
-!- newlido [~quassel@41.218.172.126] has quit [Read error: Operation timed out]16:46
lisitsyn1HeikoS: hey16:53
@HeikoSlisitsyn1: hej!16:54
lisitsyn1HeikoS: soeren submitted slots before me so we should be ok ;)16:54
@HeikoSgreeting from a Stockholm's hip coffee place :)16:54
@HeikoSok great well done16:54
lisitsyn1HeikoS: I envy you16:54
lisitsyn1HeikoS: so it sieben to acht16:56
lisitsyn1it is*16:56
@HeikoSIts quite nice here, fernando lived in this town for a while here16:56
@HeikoSyeah16:56
@HeikoSlisitsyn1: thats good right?16:56
lisitsyn1HeikoS: totally16:56
@HeikoSand once we have the slots we submit who is accepted etc?16:56
lisitsyn1HeikoS: yes we just mark them16:56
lisitsyn1and that's it16:56
lisitsyn1HeikoS: I'm currently setting up my blog :D16:58
@HeikoSlisitsyn1: cool16:59
@HeikoSwhat about?16:59
lisitsyn1well you know16:59
lisitsyn1:D16:59
lisitsyn1HeikoS: mostly some geeky details of programming some stuff17:00
@HeikoScool17:00
@HeikoSlet me know once its up17:00
@HeikoSlisitsyn1: where is the draft for the modular framework stuff?17:00
lisitsyn1HeikoS: non yet17:01
lisitsyn1there is a few unresolved issues17:01
lisitsyn1I need to be sure it is possible to do everything I wanted17:01
lisitsyn1especially swig things17:01
@HeikoSwell the draft is for discussion right? doesnt need to be perfect :)17:02
lisitsyn1ok ok I was lazy ;)17:02
@HeikoShaha17:04
@HeikoSlisitsyn1: where is your blog?17:04
lisitsyn1HeikoS: not yet ready17:09
lisitsyn1I spent quite a while setting up theme etc17:09
lisitsyn1and I need to finish two posts I had in mind17:10
@HeikoSI see17:14
@HeikoSwhat are you using?17:14
lisitsyn1HeikoS: octopress17:15
-!- newlido [~quassel@41.237.211.127] has joined #shogun17:45
-!- newlido_ [~quassel@41.237.211.127] has joined #shogun17:48
-!- HeikoS [~heiko@90-227-238-158-no15.business.telia.com] has quit [Quit: Leaving.]18:02
-!- thoralf [~thoralf@91.65.138.123] has joined #shogun18:54
thoralfHey.18:54
-!- thoralf [~thoralf@91.65.138.123] has quit [Read error: Operation timed out]19:13
-!- kislay [~kislay@210.212.61.185] has joined #shogun19:26
-!- kislay [~kislay@210.212.61.185] has quit [Read error: Connection reset by peer]19:30
-!- khalednasr [~k.nasr92@41.69.180.184] has joined #shogun19:35
-!- thoralf [~thoralf@91-65-138-123-dynip.superkabel.de] has joined #shogun19:37
-!- sonne|work [~sonnenbu@sams-office-nat.tomtomgroup.com] has quit [Ping timeout: 240 seconds]19:42
-!- sonne|work [~sonnenbu@91-64-78-194-dynip.superkabel.de] has joined #shogun19:43
-!- kislay [~kislay@14.139.41.150] has joined #shogun19:51
-!- kislay [~kislay@14.139.41.150] has quit [Read error: Connection reset by peer]19:52
-!- kislay [~kislay@14.139.41.150] has joined #shogun19:55
-!- kislay [~kislay@14.139.41.150] has quit [Read error: Connection reset by peer]19:56
-!- khalednasr [~k.nasr92@41.69.180.184] has quit [Quit: Leaving]20:14
-!- rho_ [~quassel@217.55.243.29] has joined #shogun22:07
-!- kprah [67157e4c@gateway/web/freenode/ip.103.21.126.76] has joined #shogun22:26
thoralfHey kprah22:29
kprahhey22:29
kprahcan we get the shogun root directory name from SGIO?22:30
kprahor some other file, thoralf?22:30
kprahYou are from?22:32
thoralfkprah: If you have a minimal file, then it should be possible to generate it automatically from the unit test.  So no need for getting the path.22:32
thoralfkprah: Do you know what the *minimal* parameters for this problem are?22:32
kprahthoralf: I think just a csv file with some data.22:33
kprahthoralf: Well if you see the already present unit test in tests/unit/io, there is a test already which passes.22:34
thoralfThe macro __FILE__ should be expanded to the name of the unit-test.22:35
thoralf__FILE__ ".testdata" could be used as the filename for your data.22:35
thoralfBut still, I insist that you create some *minimal* example.22:36
thoralf12 MB example of such file is not acceptable.22:36
kprahok ..... I guess I'll have to make the whole library again , since python modular is turned off :D22:37
kprahthoralf: can I get back to you after some minutes?22:37
thoralfWait.22:37
thoralfNo need to rebuild it.22:37
kprahThen, how do I turn python modular on?22:38
kprahthoralf?22:38
thoralfNo, try to analyze the problem.22:39
thoralfYou should be able to do it without running python again.22:39
thoralfJust look to the file - what caused the problem?22:39
thoralfIf you run the (old, unfixed) code on the same file, how many features/vectors do you get?22:40
kprah2000122:40
thoralfOkay.  And if you remove the trailing newline, how many will you get then?22:41
thoralfOnly the newline.  All data stays the same.22:41
kprahok let me try that22:41
kprahThere is no trailing newline here I guess. I am opening it in gedit.22:42
-!- sonne|work [~sonnenbu@91-64-78-194-dynip.superkabel.de] has quit [Ping timeout: 240 seconds]22:42
-!- sonne|work [~sonnenbu@sams-office-nat.tomtomgroup.com] has joined #shogun22:43
kprahthoralf, but for any given file how do we know what are the insides beforehand?22:43
thoralfPlease answer my question first.22:44
thoralfIf you remove the trailing newline (there is one) will the output is correct then?22:44
kprahHow do I do it, manually or by code? I am unable to see a newline after the last line.22:46
kprahthoralf?22:46
thoralfJust edit the code with your editor.22:47
thoralfSorry, the data file.22:47
-!- Netsplit *.net <-> *.split quits: lisitsyn122:49
-!- sonne|osx [~sonne@f053038174.adsl.alicedsl.de] has joined #shogun22:49
kprahyeah, same22:54
kprahthoralf, 2000122:54
thoralfOkay, and what if you create a file that has exactly one line "0.12345" and that's all?22:55
thoralfNo more lines, no more features.22:55
kprahok let me see.22:55
-!- lisitsyn [~lisitsyn@109-226-124-25.clients.tlt.100megabit.ru] has joined #shogun22:57
kprahfeatures 122:57
kprahvectors 122:57
kprahthis I created manually22:57
kprahthoralf22:57
thoralfOkay, one second.22:58
kprahokay23:01
kprahany thoughts?23:10
kprahrhoralf: sorry but can you post any advice on the mailing list, its getting late (about 3am).23:11
-!- sonne|work [~sonnenbu@sams-office-nat.tomtomgroup.com] has quit [Ping timeout: 255 seconds]23:12
kprahThanks. Bye.23:12
-!- kprah [67157e4c@gateway/web/freenode/ip.103.21.126.76] has quit [Quit: Page closed]23:12
-!- sonne|work [~sonnenbu@91-64-78-194-dynip.superkabel.de] has joined #shogun23:12
-!- newlido_ [~quassel@41.237.211.127] has quit [Remote host closed the connection]23:13
-!- newlido [~quassel@41.237.211.127] has quit [Read error: Connection reset by peer]23:14
lisitsynthoralf:23:23
lisitsynhey23:23
thoralfHey lisitsyn23:23
-!- khalednasr [~k.nasr92@41.69.190.77] has joined #shogun23:31
--- Log closed Mon Apr 07 00:00:11 2014