:py:mod:`MomentGauge.Sampler.ExponentialFamilySampler` ====================================================== .. py:module:: MomentGauge.Sampler.ExponentialFamilySampler Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: MomentGauge.Sampler.ExponentialFamilySampler.ExponentialFamilySampler MomentGauge.Sampler.ExponentialFamilySampler.ExpFamilyImportanceSampler .. py:class:: ExponentialFamilySampler(suff_stats) Bases: :py:obj:`MomentGauge.Sampler.Base.BaseSampler` The base class for exponential family sampler. A sampler of the probability distribution :math:`f(\mathbf{u};\boldsymbol{\beta})` parametrized by :math:`\boldsymbol{\beta}` from which we could draw samples and compute likelihoods. Specifically, the distribution has the form .. math:: :nowrap: \begin{equation} f(\mathbf{u};\boldsymbol{\beta},\mathbf{g}) = \exp\left( \sum_{i=0}^M \beta_i \phi_i(\mathbf{u},\mathbf{g}) \right) \end{equation} in which :math:`\{\phi_i,i=0,\cdots,M\}` are sufficient statistics, :math:`\boldsymbol{\beta}` is the natural parameter of the distribution, :math:`\phi_0(\mathbf{u},\mathbf{g}) = 1`, and :math:`\mathbf{g}` is extra gauge parameters that may or may not be requested by the moments :math:`\phi_i`. :param suff_stats: a list of moment functions [:math:`\phi_i,i=0,\cdots,M`], in which each :math:`\phi_i` is a function :math:`\phi_i` ( **u** , :math:`*` **gauge_paras** ) whose **Parameters**: **u** : float array of shape (3) - The 3D sample vector :math:`*` **gauge_paras** : - Arbitrary many extra parameters. The :math:`*` refers to the unpacking operator in python. **Returns**: float -- the moment value The lengh of the list may vary. Its first element must satisfy :math:`\phi_0` ( **u** , :math:`*` **gauge_paras** ) = 1 :type suff_stats: list .. attribute:: constant an empty dict :type: dict .. attribute:: num_statistics The number of sufficient statistics :type: int .. py:method:: suff_statistics(u, gauge_paras=()) Compute the value of sufficient statistics at certain :math:`\mathbf{u}` :param u: The 3D sample vector :type u: float array of shape (3) :param gauge_paras: A tuple ( para1, para2, ... ) containing arbitrary many extra parameters, i.e :math:`\mathbf{g}` , required by sufficient statistics :math:`\{\phi_i\}` as **gauge_paras**. Defaut is (), an empty tuple. :type gauge_paras: tuple :returns: The array :math:`\{ \phi_0(\mathbf{u}), \cdots, \phi_M(\mathbf{u}) \}` :rtype: float array of shape (M+1) .. py:method:: sample(betas, gauge_paras=()) :abstractmethod: Generate N samples :math:`\mathbf{u}_i` from the distribution :math:`f(\mathbf{u})` with proper weights :math:`w_i` such that .. math:: :nowrap: \begin{equation} \int \psi(\mathbf{u}) f(\mathbf{u};\boldsymbol{\beta},\mathbf{g}) d \mathbf{u} \approx \sum_{i=1}^N w_i(\boldsymbol{\beta},\mathbf{g}) \psi(\mathbf{u}_i (\boldsymbol{\beta},\mathbf{g}) ), \end{equation} in which :math:`\psi` is arbitrary test function, :math:`\boldsymbol{\beta}` is the natural parameter, :math:`\mathbf{g}` is extra gauge parameters that may or may not be requested, and N depends on the particular implementation of the sampler. :param betas: the natural parameter :math:`\boldsymbol{\beta}` of the distribution :type betas: float array of shape (M+1) :param gauge_paras: A tuple ( para1, para2, ... ) containing arbitrary many extra parameters, i.e :math:`\mathbf{g}` , required by sufficient statistics :math:`\{\phi_i\}` as **gauge_paras**. Defaut is (), an empty tuple. :type gauge_paras: tuple :returns: A tuple containing **samples**: *float array of shape (N,3)* - N samples of 3-dim vectors :math:`\mathbf{u}_i` draw from the distribution. **weights**: *float array of shape (N)* - N non-negative weights :math:`w_i` for each samples. **log_likelihoods**: *float array of shape (N)* - N the log-likelihoods :math:`\log f(\mathbf{u}_i)` for each samples in which N is determined by the specific implementation. :rtype: Tuple .. py:method:: LogProb(betas, u, gauge_paras=()) Calculate the log-likelihood of the distribution :math:`\log f(\mathbf{u};\boldsymbol{\beta},\mathbf{g})` at a certain sample point :math:`\mathbf{u}` :param betas: the natural parameter :math:`\boldsymbol{\beta}` of the distribution :type betas: float array of shape (M+1) :param u: the sample point :math:`\mathbf{u}` at which the likelihood is evaluated. :type u: float array of shape (3) :param gauge_paras: A tuple ( para1, para2, ... ) containing arbitrary many extra parameters, i.e :math:`\mathbf{g}` , required by sufficient statistics :math:`\{\phi_i\}` as **gauge_paras**. Defaut is (), an empty tuple. :type gauge_paras: tuple :returns: the likelihood :math:`\sum_{i=0}^M \beta_i \phi_i(\mathbf{u},\mathbf{g})` :rtype: float .. py:method:: sample_Loss(betas, moments, gauge_paras=(), base_args=()) :abstractmethod: The optimization objective loss as a function of parameters :math:`\boldsymbol{\beta}`, moments of sufficient statistics :math:`\mathbf{M}` and necessary gauge parameters. Minimizing the optimization objective given a set of moments yields the corresponding parameters :math:`\boldsymbol{\beta}`. :param betas: the natural parameter :math:`\boldsymbol{\beta}` of the distribution :type betas: float array of shape (M+1) :param moments: the target moments we wish the distribution to have as moments of sufficient statistics. :type moments: float array of shape (M+1) :param gauge_paras: A tuple ( para1, para2, ... ) containing arbitrary many extra parameters, i.e :math:`\mathbf{g}` , required by sufficient statistics :math:`\{\phi_i\}` as **gauge_paras**. Defaut is (), an empty tuple. :type gauge_paras: tuple :param base_args: A tuple ( para1, para2, ... ) containing arbitrary many parameters required by **BaseSampler.sample**. Defaut is (), an empty tuple. :type base_args: tuple :returns: the loss value :math:`L` :rtype: float .. py:class:: ExpFamilyImportanceSampler(suff_stats, baseSampler: MomentGauge.Sampler.Base.BaseSampler) Bases: :py:obj:`ExponentialFamilySampler` The sampler for exponential family by importance sampling. A sampler of the probability distribution :math:`f(\mathbf{u};\boldsymbol{\beta})` parametrized by :math:`\boldsymbol{\beta}` from which we could draw samples and compute likelihoods. Specifically, the distribution has the form .. math:: :nowrap: \begin{equation} f(\mathbf{u};\boldsymbol{\beta},\mathbf{g}) = \exp\left( \sum_{i=0}^M \beta_i \phi_i(\mathbf{u},\mathbf{g}) \right) \end{equation} in which :math:`\{\phi_i\}` are sufficient statistics, :math:`\boldsymbol{\beta}` is the natural parameter of the distribution, :math:`\phi_0(\mathbf{u},\mathbf{g}) = 1`, and :math:`\mathbf{g}` is extra gauge parameters that may or may not be requested by the moments :math:`\phi_i`. :param suff_stats: a list of moment functions [:math:`\phi_i,i=0,\cdots,M`], in which each :math:`\phi_i` is a function :math:`\phi_i` ( **u** , :math:`*` **gauge_paras** ) whose **Parameters**: **u** : float array of shape (3) - The 3D sample vector :math:`*` **gauge_paras** : - Arbitrary many extra parameters. The :math:`*` refers to the unpacking operator in python. **Returns**: float -- the moment value The lengh of the list may vary. Its first element must satisfy :math:`\phi_0` ( **u** , :math:`*` **gauge_paras** ) = 1 :type suff_stats: list :param baseSampler: The primary sampler used by importance sampling to generate samples. Its method :meth:`sample` must have benn implemented. :type baseSampler: An instance of :class:`Sampler.Base.BaseSampler` .. attribute:: constant an empty dict :type: dict .. attribute:: num_statistics The number of sufficient statistics :type: int .. py:method:: sample(betas, gauge_paras=(), base_args=()) Generate N samples :math:`\mathbf{u}_i` by importance sampling from the distribution :math:`f(\mathbf{u})` with proper weights :math:`w_i` such that .. math:: :nowrap: \begin{equation} \int \psi(\mathbf{u}) f(\mathbf{u};\boldsymbol{\beta},\mathbf{g}) d \mathbf{u} \approx \sum_{i=1}^N w_i(\boldsymbol{\beta},\mathbf{g}) \psi(\mathbf{u}_i (\boldsymbol{\beta},\mathbf{g}) ), \end{equation} in which :math:`\psi` is arbitrary test function, :math:`\boldsymbol{\beta}` is the natural parameter, :math:`\mathbf{g}` is extra gauge parameters that may or may not be requested, and N depends on the particular implementation of the sampler. :param betas: the natural parameter :math:`\boldsymbol{\beta}` of the distribution :type betas: float array of shape (M+1) :param gauge_paras: A tuple ( para1, para2, ... ) containing arbitrary many extra parameters, i.e :math:`\mathbf{g}` , required by sufficient statistics :math:`\{\phi_i\}` as **gauge_paras**. Defaut is (), an empty tuple. :type gauge_paras: tuple :param base_args: A tuple ( para1, para2, ... ) containing arbitrary many parameters required by **BaseSampler.sample**. Defaut is (), an empty tuple. :type base_args: tuple :returns: A tuple containing **samples**: *float array of shape (N,3)* - N samples of 3-dim vectors :math:`\mathbf{u}_i` draw from the distribution. **weights**: *float array of shape (N)* - N non-negative weights :math:`w_i` for each samples. **log_likelihoods**: *float array of shape (N)* - N the log-likelihoods :math:`\log f(\mathbf{u}_i)` for each samples in which N is determined by the specific implementation. :rtype: Tuple .. py:method:: sample_Loss(betas, moments, gauge_paras=(), base_args=()) Approximate the entropy loss of exponential family distribution .. math:: :nowrap: \begin{equation} \begin{split} L(\boldsymbol{\beta},\mathbf{g}, \mathbf{M}) &= \int f(\mathbf{u};\boldsymbol{\beta},\mathbf{g}) d\mathbf{u} - \sum_{i=0}^M \beta_i M_i\\ &\approx \sum_{i=1}^{N} w_i(\boldsymbol{\beta},\mathbf{g}) - \sum_{i=0}^M \beta_i M_i \end{split} \end{equation} by sampling weights :math:`w_i`, provided the natural parameters :math:`\boldsymbol{\beta}`, the moments :math:`\mathbf{M}` and necessary gauge parameters. :param betas: the natural parameter :math:`\boldsymbol{\beta}` of the distribution :type betas: float array of shape (M+1) :param moments: the target moments we wish the distribution to have as moments of sufficient statistics. :type moments: float array of shape (M+1) :param gauge_paras: A tuple ( para1, para2, ... ) containing arbitrary many extra parameters, i.e :math:`\mathbf{g}` , required by sufficient statistics :math:`\{\phi_i\}` as **gauge_paras**. Defaut is (), an empty tuple. :type gauge_paras: tuple :param base_args: A tuple ( para1, para2, ... ) containing arbitrary many parameters required by **BaseSampler.sample**. Defaut is (), an empty tuple. :type base_args: tuple :returns: the loss value :math:`L` :rtype: float .. py:method:: suff_statistics(u, gauge_paras=()) Compute the value of sufficient statistics at certain :math:`\mathbf{u}` :param u: The 3D sample vector :type u: float array of shape (3) :param gauge_paras: A tuple ( para1, para2, ... ) containing arbitrary many extra parameters, i.e :math:`\mathbf{g}` , required by sufficient statistics :math:`\{\phi_i\}` as **gauge_paras**. Defaut is (), an empty tuple. :type gauge_paras: tuple :returns: The array :math:`\{ \phi_0(\mathbf{u}), \cdots, \phi_M(\mathbf{u}) \}` :rtype: float array of shape (M+1) .. py:method:: LogProb(betas, u, gauge_paras=()) Calculate the log-likelihood of the distribution :math:`\log f(\mathbf{u};\boldsymbol{\beta},\mathbf{g})` at a certain sample point :math:`\mathbf{u}` :param betas: the natural parameter :math:`\boldsymbol{\beta}` of the distribution :type betas: float array of shape (M+1) :param u: the sample point :math:`\mathbf{u}` at which the likelihood is evaluated. :type u: float array of shape (3) :param gauge_paras: A tuple ( para1, para2, ... ) containing arbitrary many extra parameters, i.e :math:`\mathbf{g}` , required by sufficient statistics :math:`\{\phi_i\}` as **gauge_paras**. Defaut is (), an empty tuple. :type gauge_paras: tuple :returns: the likelihood :math:`\sum_{i=0}^M \beta_i \phi_i(\mathbf{u},\mathbf{g})` :rtype: float