tag:blogger.com,1999:blog-4196841648700777718.post8767041794944243342..comments2024-03-28T00:29:06.683-07:00Comments on Maximize Productivity with Industrial Engineer and Operations Research Tools: Big Data Logistic Regression with R and ODBClarrydaghttp://www.blogger.com/profile/06592231655463233562noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-4196841648700777718.post-55541976798167678902022-07-22T16:03:34.437-07:002022-07-22T16:03:34.437-07:00I enjoyed reading tthisI enjoyed reading tthisFontana Animal Controlhttps://www.animal-control-removal.com/us/animal-removal-california/fontana-animal-control.shtmlnoreply@blogger.comtag:blogger.com,1999:blog-4196841648700777718.post-1115380372803432772016-02-18T05:40:07.251-08:002016-02-18T05:40:07.251-08:00Very interesting! Thanks a lot for this nice and h...Very interesting! Thanks a lot for this nice and helpful writing piece. I'd like to welcome you to online <a href="http://writ-ing.services/" rel="nofollow">essay writing service</a> and browse around! Happy browsing! <br />paulsmith198914@gmail.comhttps://www.blogger.com/profile/01097862152097877690noreply@blogger.comtag:blogger.com,1999:blog-4196841648700777718.post-75174130794464665532013-10-16T13:23:16.608-07:002013-10-16T13:23:16.608-07:00Do you know how/if it possible to do a CONDITIONAL...Do you know how/if it possible to do a CONDITIONAL logistic regression with biglm, which uses matched case-controls such as with the function clogit in the Survival package? My data is set up for clogit in that I have all the 0s corresponding to a 1 identified as different strata. Would appreciate any thoughts on this!brianaahttps://www.blogger.com/profile/05299023381733490558noreply@blogger.comtag:blogger.com,1999:blog-4196841648700777718.post-38904631200673384712011-12-30T05:12:08.579-08:002011-12-30T05:12:08.579-08:00Thanks Louis. That is great feedback. I'll be...Thanks Louis. That is great feedback. I'll be sure to play around with that to optimize the script.larrydaghttps://www.blogger.com/profile/06592231655463233562noreply@blogger.comtag:blogger.com,1999:blog-4196841648700777718.post-36759267666315376002011-12-01T07:48:53.351-08:002011-12-01T07:48:53.351-08:00The way you've written it the data will still ...The way you've written it the data will still be read into R in full and not in chunks. This still saves memory in the calculation of the glm but does not solve the problem if your data is larger than the available memory.<br /><br />To fix this you need to pass the odbc connection object as data and specify the tablename="SQL tablename". In this process I have however discovered a bug in that the bigglm code uses sql syntax not compatable with MS SQL.Louis Rossouwhttps://www.blogger.com/profile/18329310322234338726noreply@blogger.comtag:blogger.com,1999:blog-4196841648700777718.post-34057516854392818092010-12-14T05:46:25.072-08:002010-12-14T05:46:25.072-08:00Will, its really hard to say. I can see very lar...Will, its really hard to say. I can see very large data sets being applied with RODBC and Biglm. I know a data set I'm considering is 300,000+ observations with 500+ variables. I could suspect this method can go beyond that size. Of course it could be computationally expensive and take a while to run.larrydaghttps://www.blogger.com/profile/06592231655463233562noreply@blogger.comtag:blogger.com,1999:blog-4196841648700777718.post-17416101565142507812010-12-12T11:04:35.624-08:002010-12-12T11:04:35.624-08:00I'm curious as to how large a data set you'...I'm curious as to how large a data set you'd consider running through logistic regression in R, and how large, through Biglm?Will Dwinnellhttps://www.blogger.com/profile/03379859054257561952noreply@blogger.comtag:blogger.com,1999:blog-4196841648700777718.post-79405090558028577782010-12-10T05:49:44.911-08:002010-12-10T05:49:44.911-08:00That's a good question and does require some t...That's a good question and does require some thought on how the data is prepared. I'm wondering if there are a lot of levels in the data that some pre-processing might be done to specify factors for each categorical variable. This is a good point you bring up.larrydaghttps://www.blogger.com/profile/06592231655463233562noreply@blogger.comtag:blogger.com,1999:blog-4196841648700777718.post-67842253373164805772010-12-09T19:02:36.646-08:002010-12-09T19:02:36.646-08:00I've wondered about this restriction in the do...I've wondered about this restriction in the doc, "Factors are permitted, but the levels of the factor must be the same across all data chunks (empty factor levels are ok)." <br /><br />How can you guarantee that a given chunk size will see all levels of each factor, or does the query first execute a "distinct" on each factor to allow empty levels in any chunk?Anonymousnoreply@blogger.com