Open in new window / Try shogun cloud
--- Log opened Tue Sep 25 00:00:17 2012
-!- blackburn [~blackburn@37.61.181.31] has quit [Quit: Leaving.]00:00
-!- hoijui [~hoijui@dslb-088-074-107-101.pools.arcor-ip.net] has quit [Ping timeout: 240 seconds]00:06
-!- heiko [~heiko@host86-182-162-130.range86-182.btcentralplus.com] has quit [Quit: Leaving.]00:44
-!- wiking_ [~wiking@info2k1.hu] has joined #shogun01:27
-!- voket [26688042@gateway/web/freenode/ip.38.104.128.66] has quit [Ping timeout: 245 seconds]01:30
-!- wiking [~wiking@huwico/staff/wiking] has quit [Write error: Connection reset by peer]01:30
-!- audy- [~audy@heyaudy.com] has joined #shogun01:36
-!- audy [~audy@unaffiliated/audy] has quit [Write error: Broken pipe]01:37
shogun-buildbotbuild #112 of nightly_default is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_default/builds/11203:43
shogun-buildbotbuild #113 of nightly_default is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_default/builds/11307:33
-!- hoijui [~hoijui@dslb-088-074-107-101.pools.arcor-ip.net] has joined #shogun09:04
-!- sonne|work [~sonnenbu@194.78.35.195] has joined #shogun09:05
-!- wiking_ is now known as wiking09:36
-!- wiking [~wiking@info2k1.hu] has quit [Changing host]09:36
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun09:36
-!- romi__ [~mizobe@187.74.0.20] has quit [Ping timeout: 264 seconds]09:48
-!- romi__ [~mizobe@187.57.9.8] has joined #shogun09:51
wikingsonne|work: here?09:53
sonne|workyes09:53
wikinghave u managed to get the static analyzer working?09:54
sonne|workwhat is a static analyzer?09:54
sonne|workso no :)09:54
wikingah ok09:55
wikingyou know this one: http://clang-analyzer.llvm.org/09:55
sonne|workahh09:55
wikingi'm happy to help out to make it work09:55
wikingbut i'd need access to a buildbot where there's clang09:55
wiking(i don't have on freebsd)09:55
sonne|workno totally forgot about this - please file a github issue about this09:55
wikingheheheh ok09:55
sonne|workand assign it to me09:55
wikingas said if u give me access to a bot i can make it for u :P09:56
sonne|workwell get it to work locally and I put it live :P09:56
wikingsonne|work: afaik you just need to change the ./configure and make command on the bot09:56
wikingsonne|work: at least it works for me like that09:56
sonne|workwiking: btw I heard you can now be officially called 'husband wiking' - congrats on that!09:57
wikingbut i'll put it into the github issue, how i build it09:57
sonne|workwiking: then just send me these commands and all good09:57
wikingsonne|work: ehehhehe thanx!09:57
wikingsonne|work: i'll just check now whether you can put the output somewhere else... i mean the output of the static analyzer, since it's by default in a /tmp directory09:57
sonne|workohh that would create clashes09:58
sonne|worksome local ./tmp would be better09:58
wikingyeah it has -o option09:59
wikingso it puts it wherever you want it to09:59
wikingdone10:04
wikingit seems i cannot assign10:04
wikingsonne|work: https://github.com/shogun-toolbox/shogun/issues/78910:04
wikingso i think you'll have to assign yourself10:04
sonne|workwiking: done10:13
wikingyey10:13
sonne|workwiking: btw I recall that you wanted to get google mock/test framework to work they way it was supposed10:13
sonne|workwiking: well assigned to myself10:13
sonne|worknot implemented yet10:14
wikingthe way t was supposed to?10:14
wikingah you mean the whole change about statically linking it with the shogun library?10:14
-!- blackburn [5bdfb203@gateway/web/freenode/ip.91.223.178.3] has joined #shogun10:26
blackburnuh oh ah I have an interview in 3 hours10:26
wikingblackburn: goodluckwithath10:27
blackburnheh thanks10:27
wikingblackburn: where r u heading ?:)10:27
blackburnwiking: http://wesee.com/10:27
wikingblackburn: oh cool10:29
wikingwould u move or stay in russia?10:29
blackburnno, no10:29
blackburnsame city10:29
blackburnapparently 50 meters far away from current job :D10:30
wikinglol10:30
wikingnice :)10:30
wikingbtw: i'll upload a static analyzer output again with the latest code10:30
blackburnnice10:30
wikingand hopefully sonne|work will make it automatic on one of the buildbots soon10:31
wikingdone http://maeth.com/shogun-report/10:37
wikingblackburn: slep has leaks10:39
blackburnwiking: no, not really it is bad code IIRC10:40
wikingblackburn: ?10:40
wikingblackburn: http://maeth.com/shogun-report/report-a0hTXf.html#EndPath10:40
wikingthis leaks definitely10:40
blackburnwiking: in case of malloc failure?10:41
blackburnkind of.. but securing malloc failures is something too safe for me10:42
wikingblackburn: you need to release the memory10:43
wikingif one of them failed10:43
blackburnyeah in ideal case10:53
blackburnwiking: what I meant is that if malloc failed here - all the algorithm pukes10:54
wikingblackburn: yeah true but better to be safe than sorry afaik10:54
blackburnheh right10:54
wikingit doesn't really hurt to put 2 if branches there and do a free10:54
wikingor even a goto10:54
wikingsince there's cleanup on the end of the function10:54
wikingbut this other one is a real leak imho10:56
wikinghttp://maeth.com/shogun-report/report-CTGMmV.html#EndPath10:56
wikingas infor really never released10:56
blackburnwiking: it is not my code so I didn't really checked for all that stuff :)10:58
wikingyeah i know10:58
wikingi was just saying10:58
wikingas i think it's either you or sonne|work or me who's gonna fix this :)10:58
blackburnoh who else who else10:59
wiking:>10:59
wikingmmm i still need to hack a bit the primal solver of n4nd011:00
wikingsonne|work: btw if u use the static analyzer it'll create you libshogun.13.0.dylib11:00
wikingblackburn: r u still here or on interview now?12:47
blackburnwiking: here13:18
blackburnwiking: computation mistake :D two hours more13:18
wiking:>>>13:21
wikingdo you know anything about the init_opt in StructureModel?13:21
blackburnit comes from mosek afaik13:23
blackburnor rather for mosek13:23
wikingyeah but what's it purpose and how to create it etc?13:23
wikingi've checked one example (the MUltiClassModel)13:23
blackburnwell it is not used at all in case of bmrm13:25
wikingyeah i know13:27
wikingi'm just now trying to use the primal solver by n4nd013:27
wikingthat's why i'm asking13:27
blackburnahh13:28
blackburnnot sure I can help there13:29
wikingheheh yeah i've contacted n4nd0... hopefully he'll have time to answer today13:37
wikingi've tried with the one in MulticlassSOModel13:38
wikingbut currently the biggest problem is that mosek eats up my memory13:38
wikingvirtual memory consumption of that process is about 7 gigs :S13:38
wikingwhile if i use bmrm it uses 200megs13:38
blackburnwhy do you want msoek then?13:39
wikingi was just curious13:39
blackburnI see13:45
wikingmmm something is really wrong.... 12gigs of virtual mem14:02
wikingand still nothing14:02
blackburnokay let me try to impress these guys :)14:17
-!- blackburn [5bdfb203@gateway/web/freenode/ip.91.223.178.3] has quit [Quit: Page closed]14:17
-!- audy- [~audy@heyaudy.com] has quit [Ping timeout: 256 seconds]15:02
-!- hoijui [~hoijui@dslb-088-074-107-101.pools.arcor-ip.net] has quit [Ping timeout: 245 seconds]15:03
-!- audy [~audy@heyaudy.com] has joined #shogun15:06
-!- ptizoom [4f475132@gateway/web/freenode/ip.79.71.81.50] has joined #shogun15:49
-!- heiko1 [~heiko@host86-182-162-130.range86-182.btcentralplus.com] has joined #shogun15:51
-!- hoijui [~hoijui@141.23.65.251] has joined #shogun15:55
-!- heiko2 [~heiko@host86-179-33-12.range86-179.btcentralplus.com] has joined #shogun16:49
-!- heiko1 [~heiko@host86-182-162-130.range86-182.btcentralplus.com] has quit [Ping timeout: 256 seconds]16:50
-!- ptizoom [4f475132@gateway/web/freenode/ip.79.71.81.50] has quit [Ping timeout: 245 seconds]16:52
heiko2sonne|work, sonney2k, around?17:16
heiko2or anybody else?17:16
wikingyo17:19
heiko2wiking hi17:29
heiko2I have a c++ problem17:30
heiko2I need something which behvaves as static virtual methods17:30
heiko2i.e.17:30
heiko2I have a base class with a static method17:30
heiko2and I want to overload that17:30
heiko2(base class is abstract)17:31
heiko2and when I call the method, the static method of the subclass should be executed17:31
heiko2is there any way to do that?17:31
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun17:45
n4nd0hey wiking, how is it going with structured model thing?17:46
wikingn4nd0: mmm memory problems17:52
wikingheiko2: sorry i just came back17:52
wikingheiko2: what you can do is to do a wrapper function that is a virtual17:54
heiko2I dont have an instance17:54
heiko2of the base class17:54
wikingnever?17:54
heiko2no ist abstract17:54
heiko2this has to be solved with templates somehow17:54
heiko2Ill put my hack version online soon ans ask people17:54
wikingmmm i think u can use using keyword as well17:55
heiko2yeah I read about that, but couldnt figure it out17:55
heiko2give me an hour, I will post it on github17:55
wikingi mean your problem is that foo::bar is static, foo is an abstract class17:55
wikingbut you have foo2 : public foo17:55
wikingand you want to call foo2::bar, right?17:56
wikingn4nd0: so the thing starts to grow like hell17:56
wikingn4nd0: it hasn't yet really done anything but ate 12gigs of virtual mem17:56
wikingn4nd0: mosek is that memory hungry or i'm doing something very wrong? :)17:57
heiko2wiking, yes17:57
wikingheiko2: and when u try to call foo::bar() what does it say?17:58
wikingin foo217:58
heiko2the thing is, the compiler cannot know the subclass since there is no instance which tells type17:58
-!- blackburn [~blackburn@188.168.14.212] has joined #shogun17:58
n4nd0wiking: how big are the tests you are running?17:58
blackburnthat was uhhh17:58
-!- hoijui [~hoijui@141.23.65.251] has quit [Ping timeout: 244 seconds]17:58
blackburnI did not expect 2 hour interview :D17:58
wikingblackburn: nailed it?17:59
n4nd0wiking: have you tried running the examples for the hmsvm and see if that happens there too?17:59
wikingn4nd0: no not really17:59
wikingn4nd0: i've just tried with my own17:59
blackburnwiking: no idea, I feel like I looked stupid17:59
wikingn4nd0: it only has 245 examples17:59
blackburnbut it is the machine learning job I want :(17:59
wikingblackburn: it's normal18:00
n4nd0wiking: could you take a look if it happens in the hmsvm with the large dataset?18:00
wikingn4nd0: cmdline?18:00
wikingn4nd0: i mean what should i run18:00
n4nd0wiking: one sec, let me check18:00
blackburnheiko2: static virtual is a singleton hierarchy I think18:00
n4nd0blackburn: it probably went well if it was about ml :)18:00
blackburnn4nd0: ml part was ok yes18:00
heiko2Ill post the problem, singleton wont work18:01
wikingheiko2: gist it18:01
heiko2Problem is this:18:01
wikingheiko2: but foo::bar() should work18:01
wikingregardless if it's a subclass or not18:01
heiko2static CStreamingFeatures* from_non_streaming(CFeatures* source_features)18:01
heiko2I want a static method in CStreamingFeatures (abstract)18:01
heiko2that I can give *any* CFeatures object18:01
wikingCStreamingFeatures::from_non_streaming18:02
heiko2and then the method calls the implementation in the class of the source_features object18:02
wikingin any object should work18:02
n4nd0wiking: run structure_hmsvm_mosek.py if python is ok18:02
wikingn4nd0: libshogun? :)18:02
heiko2if source_features is CDenseFeatures<T>, the method CDenseFeatures<T>::from_non_streaming(CFeatures* source_features) should be called, etc.18:03
wikingn4nd0: only python...? as i haven't compiled python interface :P18:03
n4nd0wiking: otherwise libshogun/so_hmsvm_mosek.cpp, you can modify CTwoStateModel::simulate_two_state_data if you want to make the dataset larger18:03
wikingn4nd0: mmm yeah18:06
wikingn4nd0: quite hungry :)18:06
wiking2.5g ram18:06
wikingand then suddenly drops down to 250megs18:06
wiking:)18:06
wikingand then dies with segfault18:07
wiking:)18:07
n4nd0?18:07
n4nd0that didn't seg fault some time ago :)18:08
wikingit does for me18:08
wikinglemme gdb it18:08
n4nd0in any case I see no reason why you'd like to use mosek instead of any of the bundle methods18:09
wikingn4nd0: as the other implementation that i'm comparing with is based on mosek18:09
n4nd0ok18:09
wikingit dies here18:09
wikingCStructuredLabels* out = CStructuredLabels::obtain_from_generic(sosvm->apply());18:10
n4nd0ok18:10
wikingso_hmsvm_mosek.cpp:2818:10
n4nd0it must be caused by some other recent change I think18:10
blackburndies on obtain from generic?18:11
blackburnuhuhuh18:11
blackburnor lets say uuuhhhhhhhh18:11
-!- CIA-47 [cia@cia.vc] has quit [Ping timeout: 252 seconds]18:15
wikingdoh the accuracy is shit 62.0418:15
wikingand that's only on structured labels18:16
wikingand the train and the test set is the same :D18:16
blackburnheh18:16
wikingsomething is very wrong :S18:17
n4nd0yeah, that doesn't look good18:19
wikingso n4nd0 i need that mosek :)18:20
n4nd0what do you mean?18:20
wikingthat i'd like to get your primal solver to work on with my problem18:23
wikingand that it doesn't eat 12gigs of ram18:23
n4nd0I would like very much that it works for you too :)18:28
wikingok does the so_hmsvm_mosek.cpp works for you now18:29
wikingwith the HEAD of master?18:29
n4nd0let me check if it works for me with my what I currently have here18:34
n4nd0(not updated at all probably)18:34
wikingn4nd0: which commit is that? :)18:35
n4nd0compiling...18:49
n4nd0I am stuck with this error right now18:56
n4nd0/usr/local/etc/mosek/6/tools/platform/linux32x86/bin/libiomp5.so: undefined reference to `pthread_atfork'18:56
n4nd0I'm already including -lpthread18:57
n4nd0this error always tends to appear here and I have no idea why, it drives me crazy!18:58
n4nd0wiking: can you help me with that error ^?19:15
wikingmmm19:16
wikingwhat's your compilation line?19:16
wikingthat gives this error?19:16
n4nd0clang++ -O0 so_hmsvm_mosek.cpp -I /usr/local/etc/mosek/6/tools/platform/linux32x86/h -L /usr/local/etc/mosek/6/tools/platform/linux32x86/bin/ -lshogun -lpthread -lmosek -o so_hmsvm_mosek -DUSE_MOSEK19:17
wikingaaa19:17
wikingyou dont need -lmosek19:17
n4nd0I started using clang++ instead of g++ because I didn't manage to solve this error with g++19:17
n4nd0I already tried with that I think19:17
wikingas -lmosek is 'linked' with -lshogun19:18
n4nd0same thing without -lmosek19:18
wikingso shogun.so will depend on mosek19:18
n4nd0ok19:18
n4nd0makes sense but that didn't make the fix though19:18
wikingwhere's your shogun.so?19:18
n4nd0/usr/local/bin/shogun19:18
wikingah and that's in library path?19:19
wikingbtw: can you do a file /usr/local/bin/shogun/libshogun.so19:19
wiking?19:19
wikingfile /usr/local/bin/shogun/libshogun.so19:19
wikingwhat's the output of this19:19
n4nd0LD_LIBRARY_PATH=:/usr/local/etc/mosek/6/tools/platform/linux32x86/bin19:19
n4nd0that was my LIBRARY_PAHT19:20
n4nd0so do you think I need to create this libshogun.so?19:20
wikingno19:20
n4nd0ok19:20
wiking/usr/local/bin/shogun/libshogun.so exists19:20
wikingor?19:20
n4nd0no19:21
wikingwhere is it then?19:21
n4nd0/usr/local/bin/shogun is a file19:21
wikingbut where's your dynamic library for shogun?19:21
n4nd0whereis libshogun19:21
n4nd0libshogun: /usr/local/lib/libshogun.a /usr/local/lib/libshogun.so19:21
wikingit should be somewhere under /usr/local/lib19:21
wikingok19:21
wikingfile /usr/local/lib/libshogun.so19:21
wikingwhat's the output of this19:22
n4nd0file /usr/local/lib/libshogun.so19:22
n4nd0/usr/local/lib/libshogun.so: symbolic link to `libshogun.so.12.0'19:22
wikingok19:22
wikingfile /usr/local/lib/libshogun.so.12.019:22
n4nd0/usr/local/lib/libshogun.so.12.0: ELF 32-bit LSB shared object, Intel 80386, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=0xc256cfc8209d9f09dfd579aeeacb31bff3bcf4ab, not stripped19:22
wikingmmm19:22
n4nd0so?19:26
n4nd0do you have any clue what's going on?19:26
n4nd0this lpthread is taking my live :D19:26
wikingmmm nos nog fdzlly19:27
wikingnot really :)19:27
wikingas i thought there's a problem with the arch19:27
wikingbut seems it's ok19:27
wikingso now i really have no clue19:27
wikingonly that maybe the pthread it is trying to use19:27
wikingis the x86_64 version19:27
wikingnot 32bit ...19:27
n4nd0who knows19:32
n4nd0I have to stop for dinner now19:32
n4nd0we'll see later19:32
wikingok19:33
wikingmm i dont understand this19:42
wikingrisk is 0.29558319:42
wikingstill the results are far from being good :D19:42
n4nd0yeah ... it looks a bit weird19:48
n4nd0I don't remember the exact value for the risk though19:48
n4nd0the problem is I cannot test mosek here if we don't solve this pthread problem :S19:48
wikingmmm interesting19:55
wikingclang++ -O0 -g -I/usr/include -I/usr/include/atlas so_hmsvm_mosek.cpp -lshogun -I/Users/wiking/shogun-latent-build/include -I/opt/local/include -L/Users/wiking/shogun-latent-build/lib -I/Users/wiking/mosek/6/tools/platform/osx64x86/h -o so_hmsvm_mosek19:55
wikingworks for me w/o problems19:56
n4nd0my computer has cancer19:56
n4nd0or sth19:56
n4nd0it is so annoying19:57
-!- hoijui [~hoijui@dslb-088-074-107-101.pools.arcor-ip.net] has joined #shogun19:57
wikingn4nd0: clang++ -O0 -g so_hmsvm_mosek.cpp -lshogun -I/Users/wiking/shogun-latent-build/include -o so_hmsvm_mosek20:04
wikingi mean change -I20:04
wikingwhat's the outptu20:04
n4nd0wiking: same same20:08
n4nd0clang++ -O0 -g so_hmsvm_mosek.cpp -lshogun -I/usr/local/etc/mosek/6/tools/platform/linux32x86/h -o so_hmsvm_mosek20:09
n4nd0/usr/local/etc/mosek/6/tools/platform/linux32x86/bin/libiomp5.so: undefined reference to `pthread_atfork'20:09
n4nd0clang: error: linker command failed with exit code 1 (use -v to see invocation)20:09
wikinglol20:13
wikingand i guess clang++ -O0 -g so_hmsvm_mosek.cpp -lshogun -I/usr/local/etc/mosek/6/tools/platform/linux32x86/h -o so_hmsvm_mosek -lpthread20:14
wikinggives the same20:14
wikingldd /usr/local/lib/libshogun.so20:14
wikingwhat's the output for that?20:15
wikingn4nd0: clang++ -O0 -g so_hmsvm_mosek.cpp -lshogun -I/usr/local/etc/mosek/6/tools/platform/linux32x86/h -o so_hmsvm_mosek -pthread20:15
wikingtry this20:15
wikingnot -lpthread but -pthread20:15
n4nd0wiking: http://pastebin.com/FhscaBSr20:16
n4nd0and same with the previous ones20:16
n4nd0I tried already using -pthread instead of -lpthread20:17
wiking:))20:17
wikingmmm20:17
wikingthere's pthead20:17
wikinglibpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (20:18
wikingi really wonder wtf20:18
wikingnm /lib/i386-linux-gnu/libpthread.so.0|grep atfork20:22
wikingaaaah fuck20:25
wikingtry this still20:25
wikingclang++ -O0 -g so_hmsvm_mosek.cpp -lpthread -lshogun -I/usr/local/etc/mosek/6/tools/platform/linux32x86/h -o so_hmsvm_mosek20:25
wikingnote the order of -l flags20:26
n4nd0hehe still the error20:37
n4nd0I tried changing the order before too ;)20:37
wikingnm /lib/i386-linux-gnu/libpthread.so.0|grep atfork20:39
-!- hoijui [~hoijui@dslb-088-074-107-101.pools.arcor-ip.net] has quit [Ping timeout: 252 seconds]20:46
-!- hoijui [~hoijui@dslb-088-074-107-101.pools.arcor-ip.net] has joined #shogun20:47
n4nd0nm /lib/i386-linux-gnu/libpthread.so.0|grep atfork21:16
n4nd00000bef0 t __dyn_pthread_atfork U __register_atfork@@GLIBC_2.3.221:16
n4nd00000bef0 T pthread_atfork@GLIBC_2.021:16
n4nd0wiking: is ^ looking fine?21:16
wikingnoup21:18
wikingdoes not look fine21:18
wikingn4nd0: read this21:18
wikinghttp://ryanarn.blogspot.be/2011/07/curious-case-of-pthreadatfork-on.html21:18
n4nd0I don't understand that much to be honest21:33
n4nd0wiking: do you happen to know what needs to be done?21:36
n4nd0I am not familiarized with the linking process and I get a bit lost in the post21:37
-!- blackburn [~blackburn@188.168.14.212] has quit [Ping timeout: 246 seconds]21:40
-!- heiko2 [~heiko@host86-179-33-12.range86-179.btcentralplus.com] has left #shogun []21:41
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Ping timeout: 264 seconds]21:52
-!- heiko1 [~heiko@host86-181-80-213.range86-181.btcentralplus.com] has joined #shogun22:17
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun22:21
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Ping timeout: 246 seconds]22:29
wikingmmm22:31
* wiking is porting another latent-sosvm problem.... for doublechecking bmrm results23:37
--- Log closed Wed Sep 26 00:00:17 2012