ercbk.bsky.social

ercbk.bsky.social

@ercbk.bsky.social

Github: https://github.com/ercbk Website: https://ericbook.netlify.app/ Mastodon: @erc_bk@fosstodon.org

116 followers201 following214 posts

ercbk.bsky.social
@ercbk.bsky.social

Hmmph. These turn out to be doable (esp since my NFL team sucks) for US ET people like myself — 9:30pm Sunday.

Ben Harrap@bharrap.bsky.social

Come to our monthly show and tell, where we chat through #datascience problems, projects, and passions! 🗣️ Deep learning or Just Deep Thinking: When hand-crafted features beat neural networks, and when they don't 🙋‍♂️ Stuart Szwec 🗓️ 26 Oct, 12:30pm AEDT statsoc.org.au/event-6866407 #databs #statssky

statsoc.org.auStatistical Society of Australia - Data Science SIG Monthly Show and Tell

ercbk.bsky.social reposted

Tyler Morgan-Wall
@tylermw.com

This is absolutely insane, I just rendered this effectively 3.1 billion triangles 900 frame 16 sample per pixel 960x540 animation in #RStats last night in 6 hours using the M1 GPU and #rayrender -- it would have taken a WEEK with the CPU renderer. Completely game changing for iterative work

ercbk.bsky.social reposted

CRAN Package Updates Bot
@cranberriesfeed.bsky.social

New CRAN package catchmentACS with initial version 0.6.0 #rstats https://cran.r-project.org/package=catchmentACS

cran.r-project.orgCRAN: Package catchmentACSComputes American Community Survey (ACS) estimates for the area within a given drive time of each of a set of points, such as the locations of pre-kindergarten classrooms. Such an area is called an isochrone. The drive-time areas come from a routing service, either the 'Open Source Routing Machine' ('OSRM', &lt;<a href="https://project-osrm.org/" target="_top">https://project-osrm.org/</a>&gt;) or 'openrouteservice' (&lt;<a href="https://openrouteservice.org/" target="_top">https://openrouteservice.org/</a>&gt;), and the ACS 5-year estimates of census tracts come from the Census Bureau (&lt;<a href="https://www.census.gov/data/developers.html" target="_top">https://www.census.gov/data/developers.html</a>&gt;) through the 'tidycensus' package, one state at a time. The tracts that overlap an area are combined by area weighting, which uses two weights: the share of each tract's area that lies inside, for counts and rates, and each tract's share of the overlapping area, for the medians and per-person values of three ACS tables (median household income, median home value, and per capita income). A median or per-person value from any other table is added up like a count. Counts, medians, per-person values, and five rates are returned with a margin of error at a chosen confidence level, 90 percent by default, and with a record of the settings, inputs, and package versions behind the run. The five rates are the poverty rate, the shares of households receiving Supplemental Nutrition Assistance Program (SNAP) benefits and Supplemental Security Income, the unemployment rate, and the labor force participation rate. The margins of error of counts and rates use the approximation formulas in chapter 8 of U.S. Census Bureau (2020) "Understanding and Using American Community Survey Data: What All Data Users Need to Know" &lt;<a href="https://www.census.gov/programs-surveys/acs/library/handbooks/general.html" target="_top">https://www.census.gov/programs-surveys/acs/library/handbooks/general.html</a>&gt;. For a count the formula for a sum is applied to the weighted tract estimates, with the weights treated as fixed. For a rate the default is the formula for a ratio. Its margin of error is at least as wide as that of the formula for a proportion, which the handbook gives for ratios whose numerator is part of the denominator, as it is in all five rates. The margins of error of medians and per-person values are an approximation made by the package.

ercbk.bsky.social reposted

Charlie Gao
@shikokuchuo.net

TokyoR enjoyed the world premiere of mizu today, and Mizu-chan stole the show: lock-free shared-memory parallelism for R, plus a Python binding on the same C core. Repos are open but pre-release, and the API will change. Feedback welcome on GitHub. github.com/shikokuchuo/... #RStats #Tidyverse

Charlie Gao@shikokuchuo.net

本日の #TokyoR で、mizu のワールドプレミアをお届けしました！主役はミズちゃん💧 mizu は、ロックフリーな共有メモリで、同じマシン上のRプロセス同士をつなぐパッケージです。同じCコアのPython版 pymizu もあるので、RとPythonでチャネルを共有できます。 プレリリースなのでAPIは変わるかもしれません。ぜひ試して、感想はGitHubへ！ github.com/shikokuchuo/...

ercbk.bsky.social reposted

Emil Hvitfeldt
@emilhvitfeldt.bsky.social

I have been tinkering with how to quickly modify a Quarto theme, and I think I have a decently fast solution. Introducing the Quarto Theme Tweaker Run the `Open Theme Tweaker` command in an active qmd file. Modify the settings to your liking. Save the results. github.com/EmilHvitfeld... #quarto

ercbk.bsky.social reposted

Frank Harrell
@f2harrell.bsky.social

Update to blog article on accuracy of confidence intervals for central tendency measures for asymmetric distributions. Even a triple boostrap-t interval didn't help: fharrell.com/post/aci #Statistics #StatsSky

fharrell.comMeasures of Central Tendency for an Asymmetric Distribution, and Confidence Intervals – Statistical ThinkingThere are three widely applicable measures of central tendency for general continuous distributions: the mean, median, and pseudomedian (the mode is useful for describing smooth theoretical distributi...

ercbk.bsky.social reposted

Morten Grøftehauge
@drgroftehauge.bsky.social

Do you have a favourite resource on pair programming? Please drop a link 🙏 I'd like to do some pair programming with our student worker over the summer and he is a bit avoidant. Which is common - people don't like the idea of someone looking over their shoulder and judging them. #dataBS

View full thread
ercbk.bsky.social
@ercbk.bsky.social

With a variogram model, the kriging model can be fit. I used mlr3spatiotempcv to also choose a ML model along with tuning a local kriging model. First time around I gave the ML models calendar features and the raw projected coordinates. The result is this paint brush effect on the prediction surface

ercbk.bsky.social
@ercbk.bsky.social

I decided to go w/a thin plate spline for the coordinates instead. The Catboost has a kind of a nonlinear, contour-looking surface now. The local kriging model has a zonal pattern which is due to kNN methodology and the data's spatial distribution (I think). krigeST was the better scoring model. fin

View full thread
ercbk.bsky.social
@ercbk.bsky.social

The last plot, expands the heat map to 3-D. Semi-variance now has it's own axis, and the surface is still colored according to the semi-variance values. I find the static version of this plot the most difficult to judge the models. Plotly's interactivity helps with this. It's a bear to code though.

View full thread
ercbk.bsky.social
@ercbk.bsky.social

The empirical variogram is calculated based on the data. Then a variogram model is chosen according to how well it approximates the empirical variogram. {gstat} provides 3 diagnostic plots. In the multi-line, the spread represents temporal autocorr and y-axis — spatial autocorr. Sum Metric is best.

ercbk.bsky.social
@ercbk.bsky.social

The heatmap has time on the y-axis and distance on the x-axis with cells colored by semi-variance. The noise of the empirical variogram (sample, observed) makes this more difficult to judge the models, but I'd still have to go w/the Sum-Metric (or Simple Sum Metric). (btw lowest SSE is Sum-Metric)

View full thread
ercbk.bsky.social
@ercbk.bsky.social

Hot spot analysis extends the interpretation of local measures by classifying the cluster centers. The class depends on the statistic value and which quadrant of the Moran plot the observation lands. Interpretation exs: High - cluster of high values Low-High - low value center, high value neighbors

ercbk.bsky.social
@ercbk.bsky.social

Spatio-temporal kriging is an interpolation method that models autocorrelation using a variogram. It uses semi-variance which can be interpreted inversely of spatial autocorrelation (i.e. ↑semi-variance,↓spatial autocorrelation). The value is a function of distance and in this case, also time.

View full thread
ercbk.bsky.social
@ercbk.bsky.social

As mentioned before, spatial autocorrelation tests use spatial weights created from neighbor graphs. Most measures have global and local versions. One, Moran's I, has an extension called a Moran Plot. Among other things, it labels locations with strong influence on the measure.

ercbk.bsky.social
@ercbk.bsky.social

For the local Moran's I test, units with I statistics significantly below or above the expected value indicate negative or positive autocorrelation respectively. p-values should be adjusted. Using the 0.005 threshold, p-values in darker blue shades indicate potential cluster centers of voter turnout

View full thread
ercbk.bsky.social
@ercbk.bsky.social

Spatial lags of neighbors are the number of steps away from a node. The neighbors at a particular spatial lag are that lag’s order of neighbors (e.g. 4th lag has 4th order neighbors). I was having trouble visualizing exactly what what that meant, so I thought it was finally time to try gganimate.

ercbk.bsky.social
@ercbk.bsky.social

This heatmap (tidyplots) and table (tinytable) summarize the previous map but for all nodes. For the first lag (first row) in the heatmap, the brightest yellow cell says around 16 census tracts have 6 First Order neighbors. Connectedness peaks around the 3rd lag. More explicitly shown in the table.

ercbk.bsky.social
@ercbk.bsky.social

First up is neighbor algorithms and spatial weights which represent spatial autocorrelation using relationships between nodes on a graph. Econometric regression and autocorrelation tests make use of these. This is a map of NY census tracts illustrating k-NN where all nodes have at least k neighbors.

ercbk.bsky.social reposted

Cabel Sasser
@cabel.panic.com

Here's an easter egg in the new Lego Batman that I think all of you will REALLY appreciate. It's so good, I had to make a video.

ercbk.bsky.social reposted

Ian Hussey
@ianhussey.mmmdata.io

If you have access to any the following stats software, would you be willing to run some chi squared tests on a handful of simulated datasets and send me results? I need to verify that that my R implementation of what other software does is correct. - Stata - SAS - StatXact - SPSS DM me :)

ercbk.bsky.social reposted

Tom Smith
@analyst42.bsky.social

Sooo #rstats, I built a thing... Deploying R shiny to your own cloud server is still too hard in 2026. Lightbulb.is is my answer - connect to your digital ocean account, click to create a server, and point it at a shiny GitHub repo. That's it - spend the time you save making your customers happy!

lightbulb.isLightbulb.is

ercbk.bsky.social reposted

Ben Harrap
@bharrap.bsky.social

Help pls #rstats I'm using {mice} to impute values of factor variables. My end-goal is to keep one row per obs, with the most common imputed value being retained but there are ties I could keep imputing one additional value until the tie is broken, but is there another way to go about this?

ercbk.bsky.social reposted

Jared Huling
@jaredhuling.bsky.social

I've made some big changes to the fastglm package with version 0.1.0 on cran: cran.r-project.org/package=fast... - updates that further improve speed - three new models for more complex outcome (neg binom, hurdle, zero infl) - First bias correction - big data support ** Details outlined below:

cran.r-project.orgfastglm: Fast and Stable Fitting of Generalized Linear Models using 'RcppEigen'Fits generalized linear models efficiently using 'RcppEigen'. The iteratively reweighted least squares implementation utilizes the step-halving approach of Marschner (2011) &lt;<a href="https://doi.or...