machine learning - Multinomial Logistic Regression in spark ml vs mllib -

-

spark version 2.0.0 has stated goal bring feature parity between ml , now-deprecated mllib packages.

presently ml package provides elasticnet support binary regression. obtain multinomial apparently have accept using deprecated mllib?

the downsides of using mllib:

  • it deprecated. have "why using old stuff" questions field
  • they not use ml workflow not integrate cleanly
  • for above reasons have rewrite.

is there approach available achieving one-vs-all multinomial ml package?

this answer-in-progress. there is onevsrest classifier in spark.ml.
apparently approach provide logisticregressionclassifier binary classifier - run binary version across classes , return class highest score.

update in response @zero323. here info xiangrui meng on deprecation of mllib:

switch rdd-based mllib apis maintenance mode in spark 2.0

hi all,  more year ago, in spark 1.2 introduced ml pipeline api built on top of spark sql’s dataframes. since new dataframe-based api has been developed under spark.ml package, while old rdd-based api has been developed in parallel under spark.mllib package. while easier implement , experiment new apis under new package, became harder , harder maintain both packages grew bigger , bigger. , new users confused having 2 sets of apis overlapped functions.  started recommend dataframe-based api on rdd-based api in spark 1.5 versatility , flexibility, , saw development , usage gradually shifting dataframe-based api. counting lines of scala code, 1.5 current master added ~10000 lines dataframe-based api while ~700 rdd-based api. so, gather more resources on development of dataframe-based api , users migrate on sooner, want propose switching rdd-based mllib apis maintenance mode in spark 2.0. mean exactly?  * not accept new features in rdd-based spark.mllib package, unless block implementing new features in dataframe-based spark.ml package. * still accept bug fixes in rdd-based api. * add more features dataframe-based api in 2.x series reach feature parity rdd-based api. * once reach feature parity (possibly in spark 2.2), deprecate rdd-based api. * remove rdd-based api main spark repo in spark 3.0.  though rdd-based api in de facto maintenance mode, announcement make clear , hence important both mllib developers , users. we’d appreciate feedback!  (as side note, people use “spark ml” refer dataframe-based api or entire mllib component. causes confusion. clear, “spark ml” not official name , there no plans rename mllib “spark ml” @ time.)  best, xiangrui

another update there jira , work nearing completion of may 2016 support multiclass logistic regression in spark.ml


Comments

Post a Comment

Popular posts from this blog

ios - RestKit 0.20 — CoreData: error: Failed to call designated initializer on NSManagedObject class (again) -

-
this error has shown on many times, seems. prominent answer issue seems restkit 0.20 — coredata: error: failed call designated initializer on nsmanagedobject class however, solution posted doesn't work me. let's start beginning: i want upload file server. want parse response make sure uploaded. if not, try again later. model has boolean variable called nsnumber *senttoserver purpose (in coredata, bool saved nsnumber ) . because don't know whether sending going successful, save entity coredata model first before sending via restkit. when post request successful, server returns json string this: {"id":14,"created":true} notice that, of course, id not set on coredata model yet, both id , created new informations model. here's model: @interface recording : nsmanagedobject @property (nonatomic, retain) nsstring * audiofilepath; @property (nonatomic, retain) nsdate * createdat; @property (nonatomic, retain) nsstring * deviceid; @...
Read more

java - Digest auth with Spring Security using javaconfig -

-
so i'm trying create digest authentication spring following documentation trying translate xml "requirements" in java requirements. let's have xml in docs: <bean id="digestfilter" class= "org.springframework.security.web.authentication.www.digestauthenticationfilter"> <property name="userdetailsservice" ref="jdbcdaoimpl"/> <property name="authenticationentrypoint" ref="digestentrypoint"/> <property name="usercache" ref="usercache"/> </bean> <bean id="digestentrypoint" class= "org.springframework.security.web.authentication.www.digestauthenticationentrypoint"> <property name="realmname" value="contacts realm via digest authentication"/> <property name="key" value="acegi"/> <property name="noncevalidityseconds" value="10"/> </b...
Read more

laravel - PDOException in Connector.php line 55: SQLSTATE[HY000] [1045] Access denied for user 'root'@'localhost' (using password: YES) -

-
my .env: app_env=local app_debug=true app_key=myappkey app_url=http://localhost db_connection=mysql db_host=localhost db_port=3306 db_database=gestiontaller db_username=root db_password='root' cache_driver=file session_driver=file queue_driver=sync redis_host=127.0.0.1 redis_password=null redis_port=6379 mail_driver=smtp mail_host=mailtrap.io mail_port=2525 mail_username=null mail_password=null mail_encryption=null i've tried password ", ' , without, restarting server every time. i've tried changing localhost 172.0.0.1. yes, password correct, running mysql -u root -p introducing password root in prompt can go mysql. users table: +------------------+-----------+ | user | host | +------------------+-----------+ | debian-sys-maint | localhost | | mysql.sys | localhost | | root | localhost | | root1 | localhost | +------------------+-----------+ i've added homestead.yaml this: database_port: 3306 ...
Read more