MomentGauge.Sampler.ExponentialFamilySampler#

Module Contents#

Classes#

ExponentialFamilySampler

The base class for exponential family sampler.

ExpFamilyImportanceSampler

The sampler for exponential family by importance sampling.
class MomentGauge.Sampler.ExponentialFamilySampler.ExponentialFamilySampler(suff_stats)#

Bases: MomentGauge.Sampler.Base.BaseSampler

The base class for exponential family sampler. A sampler of the probability distribution \(f(\mathbf{u};\boldsymbol{\beta})\) parametrized by \(\boldsymbol{\beta}\) from which we could draw samples and compute likelihoods. Specifically, the distribution has the form

\begin{equation} f(\mathbf{u};\boldsymbol{\beta},\mathbf{g}) = \exp\left( \sum_{i=0}^M \beta_i \phi_i(\mathbf{u},\mathbf{g}) \right) \end{equation}

in which \(\{\phi_i,i=0,\cdots,M\}\) are sufficient statistics, \(\boldsymbol{\beta}\) is the natural parameter of the distribution, \(\phi_0(\mathbf{u},\mathbf{g}) = 1\), and \(\mathbf{g}\) is extra gauge parameters that may or may not be requested by the moments \(\phi_i\).

Parameters:

suff_stats (list) –

a list of moment functions [\(\phi_i,i=0,\cdots,M\)], in which each \(\phi_i\) is a function \(\phi_i\) ( u , \(*\) gauge_paras ) whose

Parameters:

u : float array of shape (3) - The 3D sample vector

\(*\) gauge_paras : - Arbitrary many extra parameters. The \(*\) refers to the unpacking operator in python.

Returns:

float – the moment value

The lengh of the list may vary. Its first element must satisfy \(\phi_0\) ( u , \(*\) gauge_paras ) = 1

constant#

an empty dict

Type:

dict

num_statistics#

The number of sufficient statistics

Type:

int

suff_statistics(u, gauge_paras=())#

Compute the value of sufficient statistics at certain \(\mathbf{u}\)

Parameters:

  • u (float array of shape (3)) – The 3D sample vector

  • gauge_paras (tuple) – A tuple ( para1, para2, … ) containing arbitrary many extra parameters, i.e \(\mathbf{g}\) , required by sufficient statistics \(\{\phi_i\}\) as gauge_paras. Defaut is (), an empty tuple.

Returns:

The array \(\{ \phi_0(\mathbf{u}), \cdots, \phi_M(\mathbf{u}) \}\)

Return type:

float array of shape (M+1)

abstract sample(betas, gauge_paras=())#

Generate N samples \(\mathbf{u}_i\) from the distribution \(f(\mathbf{u})\) with proper weights \(w_i\) such that

\begin{equation} \int \psi(\mathbf{u}) f(\mathbf{u};\boldsymbol{\beta},\mathbf{g}) d \mathbf{u} \approx \sum_{i=1}^N w_i(\boldsymbol{\beta},\mathbf{g}) \psi(\mathbf{u}_i (\boldsymbol{\beta},\mathbf{g}) ), \end{equation}

in which \(\psi\) is arbitrary test function, \(\boldsymbol{\beta}\) is the natural parameter, \(\mathbf{g}\) is extra gauge parameters that may or may not be requested, and N depends on the particular implementation of the sampler.

Parameters:

  • betas (float array of shape (M+1)) – the natural parameter \(\boldsymbol{\beta}\) of the distribution

  • gauge_paras (tuple) – A tuple ( para1, para2, … ) containing arbitrary many extra parameters, i.e \(\mathbf{g}\) , required by sufficient statistics \(\{\phi_i\}\) as gauge_paras. Defaut is (), an empty tuple.

Returns:

A tuple containing

samples: float array of shape (N,3) - N samples of 3-dim vectors \(\mathbf{u}_i\) draw from the distribution.

weights: float array of shape (N) - N non-negative weights \(w_i\) for each samples.

log_likelihoods: float array of shape (N) - N the log-likelihoods \(\log f(\mathbf{u}_i)\) for each samples

in which N is determined by the specific implementation.

Return type:

Tuple

LogProb(betas, u, gauge_paras=())#

Calculate the log-likelihood of the distribution \(\log f(\mathbf{u};\boldsymbol{\beta},\mathbf{g})\) at a certain sample point \(\mathbf{u}\)

Parameters:

  • betas (float array of shape (M+1)) – the natural parameter \(\boldsymbol{\beta}\) of the distribution

  • u (float array of shape (3)) – the sample point \(\mathbf{u}\) at which the likelihood is evaluated.

  • gauge_paras (tuple) – A tuple ( para1, para2, … ) containing arbitrary many extra parameters, i.e \(\mathbf{g}\) , required by sufficient statistics \(\{\phi_i\}\) as gauge_paras. Defaut is (), an empty tuple.

Returns:

the likelihood \(\sum_{i=0}^M \beta_i \phi_i(\mathbf{u},\mathbf{g})\)

Return type:

float

abstract sample_Loss(betas, moments, gauge_paras=(), base_args=())#

The optimization objective loss as a function of parameters \(\boldsymbol{\beta}\), moments of sufficient statistics \(\mathbf{M}\) and necessary gauge parameters.

Minimizing the optimization objective given a set of moments yields the corresponding parameters \(\boldsymbol{\beta}\).

Parameters:

  • betas (float array of shape (M+1)) – the natural parameter \(\boldsymbol{\beta}\) of the distribution

  • moments (float array of shape (M+1)) – the target moments we wish the distribution to have as moments of sufficient statistics.

  • gauge_paras (tuple) – A tuple ( para1, para2, … ) containing arbitrary many extra parameters, i.e \(\mathbf{g}\) , required by sufficient statistics \(\{\phi_i\}\) as gauge_paras. Defaut is (), an empty tuple.

  • base_args (tuple) – A tuple ( para1, para2, … ) containing arbitrary many parameters required by BaseSampler.sample. Defaut is (), an empty tuple.

Returns:

the loss value \(L\)

Return type:

float

class MomentGauge.Sampler.ExponentialFamilySampler.ExpFamilyImportanceSampler(suff_stats, baseSampler: MomentGauge.Sampler.Base.BaseSampler)#

Bases: ExponentialFamilySampler

The sampler for exponential family by importance sampling. A sampler of the probability distribution \(f(\mathbf{u};\boldsymbol{\beta})\) parametrized by \(\boldsymbol{\beta}\) from which we could draw samples and compute likelihoods. Specifically, the distribution has the form

\begin{equation} f(\mathbf{u};\boldsymbol{\beta},\mathbf{g}) = \exp\left( \sum_{i=0}^M \beta_i \phi_i(\mathbf{u},\mathbf{g}) \right) \end{equation}

in which \(\{\phi_i\}\) are sufficient statistics, \(\boldsymbol{\beta}\) is the natural parameter of the distribution, \(\phi_0(\mathbf{u},\mathbf{g}) = 1\), and \(\mathbf{g}\) is extra gauge parameters that may or may not be requested by the moments \(\phi_i\).

Parameters:

  • suff_stats (list) –

    a list of moment functions [\(\phi_i,i=0,\cdots,M\)], in which each \(\phi_i\) is a function \(\phi_i\) ( u , \(*\) gauge_paras ) whose

    Parameters:

    u : float array of shape (3) - The 3D sample vector

    \(*\) gauge_paras : - Arbitrary many extra parameters. The \(*\) refers to the unpacking operator in python.

    Returns:

    float – the moment value

    The lengh of the list may vary. Its first element must satisfy \(\phi_0\) ( u , \(*\) gauge_paras ) = 1

  • baseSampler (An instance of Sampler.Base.BaseSampler) – The primary sampler used by importance sampling to generate samples. Its method sample() must have benn implemented.

constant#

an empty dict

Type:

dict

num_statistics#

The number of sufficient statistics

Type:

int

sample(betas, gauge_paras=(), base_args=())#

Generate N samples \(\mathbf{u}_i\) by importance sampling from the distribution \(f(\mathbf{u})\) with proper weights \(w_i\) such that

\begin{equation} \int \psi(\mathbf{u}) f(\mathbf{u};\boldsymbol{\beta},\mathbf{g}) d \mathbf{u} \approx \sum_{i=1}^N w_i(\boldsymbol{\beta},\mathbf{g}) \psi(\mathbf{u}_i (\boldsymbol{\beta},\mathbf{g}) ), \end{equation}

in which \(\psi\) is arbitrary test function, \(\boldsymbol{\beta}\) is the natural parameter, \(\mathbf{g}\) is extra gauge parameters that may or may not be requested, and N depends on the particular implementation of the sampler.

Parameters:

  • betas (float array of shape (M+1)) – the natural parameter \(\boldsymbol{\beta}\) of the distribution

  • gauge_paras (tuple) – A tuple ( para1, para2, … ) containing arbitrary many extra parameters, i.e \(\mathbf{g}\) , required by sufficient statistics \(\{\phi_i\}\) as gauge_paras. Defaut is (), an empty tuple.

  • base_args (tuple) – A tuple ( para1, para2, … ) containing arbitrary many parameters required by BaseSampler.sample. Defaut is (), an empty tuple.

Returns:

A tuple containing

samples: float array of shape (N,3) - N samples of 3-dim vectors \(\mathbf{u}_i\) draw from the distribution.

weights: float array of shape (N) - N non-negative weights \(w_i\) for each samples.

log_likelihoods: float array of shape (N) - N the log-likelihoods \(\log f(\mathbf{u}_i)\) for each samples

in which N is determined by the specific implementation.

Return type:

Tuple

sample_Loss(betas, moments, gauge_paras=(), base_args=())#

Approximate the entropy loss of exponential family distribution

\begin{equation} \begin{split} L(\boldsymbol{\beta},\mathbf{g}, \mathbf{M}) &= \int f(\mathbf{u};\boldsymbol{\beta},\mathbf{g}) d\mathbf{u} - \sum_{i=0}^M \beta_i M_i\\ &\approx \sum_{i=1}^{N} w_i(\boldsymbol{\beta},\mathbf{g}) - \sum_{i=0}^M \beta_i M_i \end{split} \end{equation}

by sampling weights \(w_i\), provided the natural parameters \(\boldsymbol{\beta}\), the moments \(\mathbf{M}\) and necessary gauge parameters.

Parameters:

  • betas (float array of shape (M+1)) – the natural parameter \(\boldsymbol{\beta}\) of the distribution

  • moments (float array of shape (M+1)) – the target moments we wish the distribution to have as moments of sufficient statistics.

  • gauge_paras (tuple) – A tuple ( para1, para2, … ) containing arbitrary many extra parameters, i.e \(\mathbf{g}\) , required by sufficient statistics \(\{\phi_i\}\) as gauge_paras. Defaut is (), an empty tuple.

  • base_args (tuple) – A tuple ( para1, para2, … ) containing arbitrary many parameters required by BaseSampler.sample. Defaut is (), an empty tuple.

Returns:

the loss value \(L\)

Return type:

float

suff_statistics(u, gauge_paras=())#

Compute the value of sufficient statistics at certain \(\mathbf{u}\)

Parameters:

  • u (float array of shape (3)) – The 3D sample vector

  • gauge_paras (tuple) – A tuple ( para1, para2, … ) containing arbitrary many extra parameters, i.e \(\mathbf{g}\) , required by sufficient statistics \(\{\phi_i\}\) as gauge_paras. Defaut is (), an empty tuple.

Returns:

The array \(\{ \phi_0(\mathbf{u}), \cdots, \phi_M(\mathbf{u}) \}\)

Return type:

float array of shape (M+1)

LogProb(betas, u, gauge_paras=())#

Calculate the log-likelihood of the distribution \(\log f(\mathbf{u};\boldsymbol{\beta},\mathbf{g})\) at a certain sample point \(\mathbf{u}\)

Parameters:

  • betas (float array of shape (M+1)) – the natural parameter \(\boldsymbol{\beta}\) of the distribution

  • u (float array of shape (3)) – the sample point \(\mathbf{u}\) at which the likelihood is evaluated.

  • gauge_paras (tuple) – A tuple ( para1, para2, … ) containing arbitrary many extra parameters, i.e \(\mathbf{g}\) , required by sufficient statistics \(\{\phi_i\}\) as gauge_paras. Defaut is (), an empty tuple.

Returns:

the likelihood \(\sum_{i=0}^M \beta_i \phi_i(\mathbf{u},\mathbf{g})\)

Return type:

float