Add Multivariate Distance Covariance metric and the corresponding test of independence by Palash123-4 · Pull Request #62 · vnmabus/dcor

Palash123-4 · 2024-01-25T20:28:39Z

As we discussed, I have added the code corresponding to the random projection based multivariate distance covariance, as discussed in https://www.frontiersin.org/articles/10.3389/fams.2021.779841/full.
I spent some time regarding how to use the gamma function in the best possible way so that the correct p_value can be calculated. (One need to install the "mpmath" python module for gamma_cdf function, for details: https://mpmath.org/doc/current/functions/expintegrals.html)
Please take a look and let me know whether it is working fine or not.

vnmabus

Thank you for your effort making this PR! I did not check the math yet (I assume you implemented it well), but I have several code suggestions.

dcor/_dcor.py

vnmabus · 2024-01-26T09:29:02Z

dcor/_dcor.py

+np.bool = np.bool_
+
+
+def gamma_ratio(p): return np.exp(gammaln((p+1)/2) - gammaln(p/2))  # For Calculating C_p and C_q


The function body needs to be in its own line.

Functions should be type-annotated.

I have added the type-annotation

Maybe I was not very clear: I meant to use Python type hints, not to add the types in the docstring. Types in a docstring cannot be easily verified, while type hints can.

I have made the changes

dcor/_dcor.py

dcor/independence.py

based distance covariance

…velop

Palash123-4 · 2024-02-01T11:38:49Z

I found some issues regarding the cyclic recalling of rowwise module in _dcor.py. Maybe you can help me to solve it.

vnmabus · 2024-02-02T09:37:01Z

dcor/_dcor.py

+    return X_new
+
+
+def u_dist_cov_sqr_mv(X, Y, n_projs = 500, method ='mergesort'):


Suggested change

def u_dist_cov_sqr_mv(X, Y, n_projs = 500, method ='mergesort'):

def u_dist_cov_sqr_mv(X, Y, n_projs = 500, method ="mergesort"):

Please, add double quotes everywhere.

I made the changes

vnmabus · 2024-02-02T09:38:35Z

dcor/_dcor.py

+
+
+"""
+A Statistically and Numerically Efficient Independence Test Based on


The docstring needs to be inside the corresponding function.

I completed the task

vnmabus · 2024-02-02T09:39:41Z

dcor/_dcor.py

+
+References
+----------
+.. bibliography:: ../refs.bib


Use :footcite: and .. footbibliography instead, as it is easier to maintain.

I would really appreciate it if you kindly guide me how to do that, I did search this names on the internet, but I am not quite certain how to do it. It would be really nice if you guide me step-by-step process.

Just use :footcite: instead of :cite: inside the text to place the footnote, and .. footbibliography:: instead of .. biblioggraphy::. You do not need to specify the .bib file, as it is done in the global config.

You can use this docstring as an example:
https://github.com/GAA-UAM/scikit-fda/blob/3eb8ace81220011559df83be0ffb9154a11fb001/skfda/exploratory/stats/_fisher_rao.py#L66-L99

I implemented the exact format that you have mentioned

dcor/_dcor.py

vnmabus · 2024-02-02T09:44:10Z

dcor/_dcor.py

+np.bool = np.bool_
+
+
+def gamma_ratio(p): return np.exp(gammaln((p+1)/2) - gammaln(p/2))  # For Calculating C_p and C_q


Maybe I was not very clear: I meant to use Python type hints, not to add the types in the docstring. Types in a docstring cannot be easily verified, while type hints can.

vnmabus · 2024-02-02T09:45:45Z

dcor/_dcor.py

+@njit(fastmath=True, parallel=True, cache=True)
+def dist_sum(X):  # It is nothing but the parallelized version of pdist function-----------------------
+    '''
+    Parameters


You closed the conversation without changing it, please don't do that.

vnmabus · 2024-02-02T09:50:10Z

dcor/_dcor.py

+    '''
+
+    # X_std = multivariate_normal.rvs( np.zeros(dim_X), np.identity(dim_X), size = 1)
+    X_std = np.random.standard_normal(dim_X)


I know what random projections are. What I meant, is that it should have a random_state argument, similar to how distance_covariance_test works. This way, it is still possible to generate random data, but you can also pass a fixed seed to obtain reproducible results.

vnmabus · 2024-02-02T09:55:45Z

dcor/_dcor.py

+    '''
+
+    # X_std = multivariate_normal.rvs( np.zeros(dim_X), np.identity(dim_X), size = 1)
+    X_std = np.random.standard_normal(dim_X)


Again, do not resolve conversations until the comments have been addressed.

vnmabus · 2024-02-02T09:57:52Z

dcor/_dcor.py

+    X_std = np.random.standard_normal(dim_X)
+    X_norm = np.sum([i**2 for i in X_std])
+    U_sphere = np.array(X_std / np.sqrt(X_norm))
+    if dim_X > 1:


Maybe, but does the general code not work in that case?

vnmabus · 2024-05-06T20:59:48Z

Note that the last commit shows a lot of whitespace changes. Please, make sure that your editor is not changing the Unix newlines (LF, or "\n") to Windows ones (CRLF, or "\r\n").

Palash123-4 · 2024-05-07T15:44:52Z

Note that the last commit shows a lot of whitespace changes. Please, make sure that your editor is not changing the Unix newlines (LF, or "\n") to Windows ones (CRLF, or "\r\n").

I use actually "spyder", not sure why it did happen. But I can use "autopipe" for better indentation if it's necessary.
Can you please tell me why am I getting this compilation error?

Removed White space

Palash123-4 · 2024-08-02T15:17:48Z

Note that the last commit shows a lot of whitespace changes. Please, make sure that your editor is not changing the Unix newlines (LF, or "\n") to Windows ones (CRLF, or "\r\n").

I have removed the white space.

Palash123-4 added 3 commits January 25, 2024 21:10

Add files via upload

b7b0653

Add the multivariate distance covariance

68a7cfa

Update _dcor.py

af0568d

vnmabus requested changes Jan 26, 2024

View reviewed changes

Palash123-4 and others added 15 commits January 31, 2024 16:36

Merge branch 'vnmabus:develop' into develop

21f4b8a

update random projection-

d44fb7d

based distance covariance

update random projection based test of independence

946397c

add dist_sum function

62152ef

Merge branch 'develop' of https://github.com/Palash123-4/dcor into de…

3207181

…velop

add citation of random projections based distance covariance

3b6fde1

update with spaces

8890215

update with spaces

0b73538

update with spaces

19a71f6

insert .

db98194

insert .

d1b4050

insert .

a41bdb8

insert .

0496e5a

update module

0718f7e

update module

b25130d

Aakash Chowdhury added 2 commits February 1, 2024 21:13

update dimension evaluation

7b55726

update dimension evaluation

c89131e

vnmabus requested changes Feb 10, 2024

View reviewed changes

Add files via upload

974c84e

Add files via upload

3d25d06

Removed White space

Merge branch 'vnmabus:develop' into develop

bb32079

		np.bool = np.bool_


		def gamma_ratio(p): return np.exp(gammaln((p+1)/2) - gammaln(p/2)) # For Calculating C_p and C_q

		return X_new


		def u_dist_cov_sqr_mv(X, Y, n_projs = 500, method ='mergesort'):



		"""
		A Statistically and Numerically Efficient Independence Test Based on

Conversation

Palash123-4 commented Jan 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vnmabus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Palash123-4 commented Feb 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vnmabus commented May 6, 2024

Uh oh!

Palash123-4 commented May 7, 2024

Uh oh!

Palash123-4 commented Aug 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Palash123-4 commented Jan 25, 2024 •

edited

Loading

Palash123-4 commented Feb 1, 2024 •

edited

Loading