Re: Bayesian binary item response theory models using bayesmh

Dear Billy,

See our answers to each of your questions below.

1. Rasch model

Your specification of the Rasch model is correct. That is, for our math and science data example, a Rasch model can be specified as

  . bayesmh y = ({subj:}-{delta:}), likelihood(logit)
>  redefine(subj:i.id) redefine(delta:i.item) ...

Here, we labelled the item-specific parameters as "delta" instead of "diff" as in our 1PL example to emphasize that the estimates from this Rasch model will be different than those from the fitted 1PL model.

There is also more details about how to fit a Rasch model using
bayesmh and its link to 1PL IRT model in example 28 in [BAYES] bayesmh.

2. Joint estimation of person- and item-specific parameters

bayesmh does estimate the person-specific and item-specific parameters jointly. In our IRT examples, we were not interested in the person-specific estimates so we used the exclude() option to exclude them from final results. If you do not use this option, the person-specific estimates will be saved with all MCMC estimates and will be displayed in the estimation table.

3. Outfit/infit statistics and residuals

Within Bayesian framework, model fit statistics are obtained by using a
so-called posterior predictive distribution, the distribution of the outcome Y given the observed data y. A posterior predictive p-value associated with a statistic of interest is often used to access model fit. We will consider writing a follow-up blog entry about Bayesian posterior predictive assessment of IRT models.

4. Many-facet Rasch models and multidimensional IRT models

By viewing many-facet Rasch models as having additional "random-effects" parameters, we can extend the basic specification by simply adding more random-effects terms.

Continuing our example of a Rasch model, suppose that there is another "facet" represented by a variable task in the dataset.

id task item q
-------------------
1 1 1 0
1 1 2 1
...
100 10 5 0

We simply add the random-effects parameters associated with task to our model specification as follows:

  . fvset base none id task item
  . bayesmh q = ({subj:}+{task:}-{delta:}), likelihood(logit)
>     redefine(subj:i.id)
>     redefine(task:i.task)
>     redefine(delta:i.item) ...

Within the IRT context, the corresponding 2-dimensional IRT model could be fit as follows. (We use the specification of a multidimensional IRT model given by formula (3) in Reckase (2007, p. 612).)

  . fvset base none id task item
  . bayesmh q = ({a1}*{subj:}+{a2}*{task:}+{d:}), likelihood(logit)       
>     redefine(subj:i.id)
>     redefine(task:i.task)
>     redefine(d:i.item) ...

where parameters {a1} and {a2} are common across items. If we wanted to make these parameters item-specific, we can use the following specification:

  . fvset base none id task item
  . bayesmh q = ({a1:}*{subj:}+{a2:}*{task:}+{d:}), likelihood(logit)       
>     redefine(subj:i.id)
>     redefine(task:i.task)
>     redefine(d:i.item)
>     redefine(a1:i.item)
>     redefine(a2:i.item) ...

You can extend models above in a straightforward way to accommodate more facets or dimensions.

Reference:

Reckase, M. D. 2007. Multidimensional Item Response Theory. In Vol. 26 of Handbook of Statistics: Psychometrics, ed. C. R. Rao and S. Sinharay, 607-642. Amsterdam: Elseiver.

-- Nikolay and Yulia