Prev: Dynamically allocate space for mxArray (C Mex)
Next: Fitting Pareto distribution CDF to ECDF of data
From: Avinash on 26 Jul 2010 12:07
Hello,

I am a big fan of the bsxfun, cellfun, etc. functions included in matlab sometime ago. I was just wondering if it would be easy to implement some extensions that would be useful in many situations. Essentially, this would involve removing the current restriction on the function argument to bsxfun that it return a column vector *of the same size* as the input.

To elaborate, let:
A = m x n matrix where m and n are large values
v = m x 1 column vector
Suppose I want to create a k x n matrix B such that B(:,ii) = fun(A(:,ii), v), where fun is a function that takes two m x 1 vectors as arguments and returns a k x 1 vector. One simple way to achieve this is:
temp_v = zeros(m-k, 1);
temp_fun = @(x, y) ([ fun(x, y) ; temp_v]); % Just pad with extra zeros to ensure same size as input
temp_B = bsxfun(temp_fun, A, v);
B = temp_B(1:k, :);

I would expect this to be reasonably time-efficient. However, when m and n are large and especially when m >> k, this seems like a waste of memory resources. Another option would be to split A into a cell array using mat2cell, call cellfun, and then use cell2mat again, but I doubt whether that would be more efficient. (The for loop option is obviously possible, but lets avoid for loops).

Ideally I would like a function, lets say colfun(fun, A, v), which would allow fun to take two column vectors (or a column vector and a scalar) and output a vector of any size. colfun would then apply fun to every column of A and v.

Is there anything intrinsic about the way bsxfun is implemented that would make this difficult to achieve ?

Thanks,
Avinash
From: us on 26 Jul 2010 12:35
"Avinash " <avinash_varna_remove_this(a)yahoo.com> wrote in message <i2kbr8$6gg$1(a)fred.mathworks.com>...
> Hello,
>
> I am a big fan of the bsxfun, cellfun, etc. functions included in matlab sometime ago. I was just wondering if it would be easy to implement some extensions that would be useful in many situations. Essentially, this would involve removing the current restriction on the function argument to bsxfun that it return a column vector *of the same size* as the input.
>
> To elaborate, let:
> A = m x n matrix where m and n are large values
> v = m x 1 column vector
> Suppose I want to create a k x n matrix B such that B(:,ii) = fun(A(:,ii), v), where fun is a function that takes two m x 1 vectors as arguments and returns a k x 1 vector. One simple way to achieve this is:
> temp_v = zeros(m-k, 1);
> temp_fun = @(x, y) ([ fun(x, y) ; temp_v]); % Just pad with extra zeros to ensure same size as input
> temp_B = bsxfun(temp_fun, A, v);
> B = temp_B(1:k, :);
>
> I would expect this to be reasonably time-efficient. However, when m and n are large and especially when m >> k, this seems like a waste of memory resources. Another option would be to split A into a cell array using mat2cell, call cellfun, and then use cell2mat again, but I doubt whether that would be more efficient. (The for loop option is obviously possible, but lets avoid for loops).
>
> Ideally I would like a function, lets say colfun(fun, A, v), which would allow fun to take two column vectors (or a column vector and a scalar) and output a vector of any size. colfun would then apply fun to every column of A and v.
>
> Is there anything intrinsic about the way bsxfun is implemented that would make this difficult to achieve ?
>
> Thanks,
> Avinash

rather than using so many words:
can you come up with a clean, small ML input/output example...

us
From: Matt J on 26 Jul 2010 14:09
"Avinash " <avinash_varna_remove_this(a)yahoo.com> wrote in message <i2kbr8$6gg$1(a)fred.mathworks.com>...

> Is there anything intrinsic about the way bsxfun is implemented that would make this difficult to achieve ?
================

Yes. bsxfun is designed to implement element-wise operations. As such, it doesn't have to distinguish between a transformation of the columns of A or of its rows. It also means that the output data size is predictable in advance, which facilitates memory allocation.

Neither of these are true with your colfun(), however. You would need some way of passing this info in advance.
From: Steven_Lord on 26 Jul 2010 14:17


"Avinash " <avinash_varna_remove_this(a)yahoo.com> wrote in message
news:i2kbr8$6gg$1(a)fred.mathworks.com...
> Hello,
>
> I am a big fan of the bsxfun, cellfun, etc. functions included in matlab
> sometime ago. I was just wondering if it would be easy to implement some
> extensions that would be useful in many situations. Essentially, this
> would involve removing the current restriction on the function argument to
> bsxfun that it return a column vector *of the same size* as the input.
>
> To elaborate, let:
> A = m x n matrix where m and n are large values
> v = m x 1 column vector
> Suppose I want to create a k x n matrix B such that B(:,ii) = fun(A(:,ii),
> v), where fun is a function that takes two m x 1 vectors as arguments and
> returns a k x 1 vector. One simple way to achieve this is:
> temp_v = zeros(m-k, 1);
> temp_fun = @(x, y) ([ fun(x, y) ; temp_v]); % Just pad with extra zeros
> to ensure same size as input
> temp_B = bsxfun(temp_fun, A, v);
> B = temp_B(1:k, :);
>
> I would expect this to be reasonably time-efficient. However, when m and n
> are large and especially when m >> k, this seems like a waste of memory
> resources. Another option would be to split A into a cell array using
> mat2cell, call cellfun, and then use cell2mat again, but I doubt whether
> that would be more efficient. (The for loop option is obviously possible,
> but lets avoid for loops).
>
> Ideally I would like a function, lets say colfun(fun, A, v), which would
> allow fun to take two column vectors (or a column vector and a scalar) and
> output a vector of any size. colfun would then apply fun to every column
> of A and v.
>
> Is there anything intrinsic about the way bsxfun is implemented that would
> make this difficult to achieve ?

You could probably do this using ARRAYFUN (or a simple FOR loop), but if
you're asking for a built-in function to do this type of operation I don't
think (if I understand your question correctly) such a built-in function
exists. It sounds like a reasonable enhancement request to file with
Technical Support, though. If you do choose to submit this request via
Technical Support, it would be useful if you could give an example of a
scenario in which you would use this function, so that the developers have a
"real" use case for this functionality to consider.

--
Steve Lord
slord(a)mathworks.com
comp.soft-sys.matlab (CSSM) FAQ: http://matlabwiki.mathworks.com/MATLAB_FAQ
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

From: Avinash on 26 Jul 2010 15:29
Here is a toy example of what I meant:

===============
file fun.m:
--------------
function out = fun(in1, in2)

y = in1 + in2;
out = [sum(y); sum(y.^2); sum(y.^3)];
===============

===============
file script.m:
-----------------

clear;
clc;

m = 1000;
n = 10000;
k = 3;
A = rand(m, n);
v = rand(m, 1);

%% For loop

disp('Executing for loop...');
tStart1 = tic;
B1 = zeros(k, n);
for ii = 1 : n
B1(:,ii) = fun(A(:,ii), v);
end
toc(tStart1);

%% Using bsxfun

disp('Using bsxfun and anonymous function ...');
t = zeros(m-k, 1);
temp_fun = @(x,y) [fun(x, y); t];
tStart2 = tic;
temp = bsxfun(temp_fun, A, v);
B2 = temp(1:k,:);
% B2 = bsxfun(@fun, A, v);
toc(tStart2)

%% Alternative approach

disp('bsxfun + computations ...');
tStart3 = tic;
y = bsxfun(@plus, A, v);
B3 = [sum(y); sum(y.^2); sum(y.^3)];
toc(tStart3);

=========================

Running script.m on my machine produces:

Executing for loop...
Elapsed time is 2.031771 seconds.
Using bsxfun and anonymous function ...
Elapsed time is 2.201835 seconds.
bsxfun + computations ...
Elapsed time is 1.221083 seconds.

My suggestion was to have a built-in function that could essentially replace the for-loop in the first cell in the hope that a built-in function could be more efficient.

@us,
Hope that clarifies what I meant.

@Matt,
While I understand that bsxfun is designed to operate element-wise, it can still be used for functions that do not operate element-wise, but produce an output of the same size as the input. In the above example, if I change fun to:
-----------------
function out = fun1(in1, in2)
out = sum(in1+in2)*ones(size(in1));
-----------------
then bsxfun(@fun1, A, v) is still (slightly) faster than the for loop.

@Steven,
Thanks. My original intention was to submit a request, but I just wanted to make sure that what I was requesting was reasonable and not already out there. I think that such a function can be useful in many scenarios.

Regards,
Avinash
 |  Next  |  Last
Pages: 1 2
Prev: Dynamically allocate space for mxArray (C Mex)
Next: Fitting Pareto distribution CDF to ECDF of data