Very slow execution when calling function
I have two functions. The second function uses the output from the first function.
One is:
DELIMITER $$
DROP FUNCTION IF EXISTS fp_splitfactor;
CREATE FUNCTION fp_splitfactor_price (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
SELECT IFNULL(EXP(SUM(LOG(f.p_split_factor))),1) INTO splitfactor
FROM fp_v2_fp_basic_splits AS f
WHERE f.fsym_id = id AND f.p_split_date > startdate AND f.p_split_date < NOW();
RETURN splitfactor;
END$$
DELIMiTER ;
Second one is:
DELIMITER $$
DROP FUNCTION IF EXISTS fp_splitadjprice;
CREATE FUNCTION fp_splitadjprice (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
DECLARE splitadjprice FLOAT;
DECLARE spinofffactor FLOAT;
SET splitfactor = 1.0;
SELECT fp_splitfactor(id, startdate) INTO splitfactor;
SELECT (p_price * splitfactor) INTO splitadjprice
FROM fp_v2_fp_basic_prices
WHERE fsym_id = id AND p_date = startdate;
RETURN splitadjprice;
END$$
DELIMITER ;
I then try to exectute a query as the following:
SELECT
p.fsym_id,
b.p_co_sec_name_desc AS Company_Name,
b.region AS Region,
p.p_date,
p.p_price AS Unadjusted_Price,
fp_splitadjprice(p.fsym_id,p_date) AS Adjusted_Price
FROM
fp_v2_fp_basic_prices p
LEFT JOIN (
SELECT r2.region, b2.p_co_sec_name_desc, b2.fsym_id
FROM fp_v2_fp_sec_coverage b2
LEFT JOIN sym_v1_sym_region r2 ON b2.fsym_id = r2.fsym_id
WHERE r2.region = "EUR") b
ON b.fsym_id =p.fsym_id
So basically my query calls the second function, which then calls the first function in order to return a value to the query. The execution is extremely slow though, but I do not understand why that is the case?
mysql mysql-workbench
|
show 1 more comment
I have two functions. The second function uses the output from the first function.
One is:
DELIMITER $$
DROP FUNCTION IF EXISTS fp_splitfactor;
CREATE FUNCTION fp_splitfactor_price (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
SELECT IFNULL(EXP(SUM(LOG(f.p_split_factor))),1) INTO splitfactor
FROM fp_v2_fp_basic_splits AS f
WHERE f.fsym_id = id AND f.p_split_date > startdate AND f.p_split_date < NOW();
RETURN splitfactor;
END$$
DELIMiTER ;
Second one is:
DELIMITER $$
DROP FUNCTION IF EXISTS fp_splitadjprice;
CREATE FUNCTION fp_splitadjprice (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
DECLARE splitadjprice FLOAT;
DECLARE spinofffactor FLOAT;
SET splitfactor = 1.0;
SELECT fp_splitfactor(id, startdate) INTO splitfactor;
SELECT (p_price * splitfactor) INTO splitadjprice
FROM fp_v2_fp_basic_prices
WHERE fsym_id = id AND p_date = startdate;
RETURN splitadjprice;
END$$
DELIMITER ;
I then try to exectute a query as the following:
SELECT
p.fsym_id,
b.p_co_sec_name_desc AS Company_Name,
b.region AS Region,
p.p_date,
p.p_price AS Unadjusted_Price,
fp_splitadjprice(p.fsym_id,p_date) AS Adjusted_Price
FROM
fp_v2_fp_basic_prices p
LEFT JOIN (
SELECT r2.region, b2.p_co_sec_name_desc, b2.fsym_id
FROM fp_v2_fp_sec_coverage b2
LEFT JOIN sym_v1_sym_region r2 ON b2.fsym_id = r2.fsym_id
WHERE r2.region = "EUR") b
ON b.fsym_id =p.fsym_id
So basically my query calls the second function, which then calls the first function in order to return a value to the query. The execution is extremely slow though, but I do not understand why that is the case?
mysql mysql-workbench
your query call recursively other query .. this could be the reason for the slow execution .. looking to the code .. could be that you can get your result using only query an not function (that recursively perform query ) .. you should add a proper (simple) data sample and the expected resultt ..
– scaisEdge
Nov 14 '18 at 17:23
@scaisEdge hmm, I will try to see if I can add a dataset. But I can't be the first one who has called functions from functions in MySQL before I reckon?
– Chris
Nov 14 '18 at 17:50
Yes you are not the firts .. nested function are normally used but if for each row you select you invoke a function that perform select .. and so on .. the performance can't be good..In SQL an approach based on the Set Theory is usually better that an algortmic approach ..
– scaisEdge
Nov 14 '18 at 17:53
Well, when I perform a query calling only 1 function it is also extremely slow. This function has a datetime value though, but it is slow even when I limit the output to 10 rows. That does not make sense to at all?
– Chris
Nov 14 '18 at 18:03
so is only the query at the bottom of the question that is low .. ? ..
– scaisEdge
Nov 14 '18 at 18:48
|
show 1 more comment
I have two functions. The second function uses the output from the first function.
One is:
DELIMITER $$
DROP FUNCTION IF EXISTS fp_splitfactor;
CREATE FUNCTION fp_splitfactor_price (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
SELECT IFNULL(EXP(SUM(LOG(f.p_split_factor))),1) INTO splitfactor
FROM fp_v2_fp_basic_splits AS f
WHERE f.fsym_id = id AND f.p_split_date > startdate AND f.p_split_date < NOW();
RETURN splitfactor;
END$$
DELIMiTER ;
Second one is:
DELIMITER $$
DROP FUNCTION IF EXISTS fp_splitadjprice;
CREATE FUNCTION fp_splitadjprice (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
DECLARE splitadjprice FLOAT;
DECLARE spinofffactor FLOAT;
SET splitfactor = 1.0;
SELECT fp_splitfactor(id, startdate) INTO splitfactor;
SELECT (p_price * splitfactor) INTO splitadjprice
FROM fp_v2_fp_basic_prices
WHERE fsym_id = id AND p_date = startdate;
RETURN splitadjprice;
END$$
DELIMITER ;
I then try to exectute a query as the following:
SELECT
p.fsym_id,
b.p_co_sec_name_desc AS Company_Name,
b.region AS Region,
p.p_date,
p.p_price AS Unadjusted_Price,
fp_splitadjprice(p.fsym_id,p_date) AS Adjusted_Price
FROM
fp_v2_fp_basic_prices p
LEFT JOIN (
SELECT r2.region, b2.p_co_sec_name_desc, b2.fsym_id
FROM fp_v2_fp_sec_coverage b2
LEFT JOIN sym_v1_sym_region r2 ON b2.fsym_id = r2.fsym_id
WHERE r2.region = "EUR") b
ON b.fsym_id =p.fsym_id
So basically my query calls the second function, which then calls the first function in order to return a value to the query. The execution is extremely slow though, but I do not understand why that is the case?
mysql mysql-workbench
I have two functions. The second function uses the output from the first function.
One is:
DELIMITER $$
DROP FUNCTION IF EXISTS fp_splitfactor;
CREATE FUNCTION fp_splitfactor_price (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
SELECT IFNULL(EXP(SUM(LOG(f.p_split_factor))),1) INTO splitfactor
FROM fp_v2_fp_basic_splits AS f
WHERE f.fsym_id = id AND f.p_split_date > startdate AND f.p_split_date < NOW();
RETURN splitfactor;
END$$
DELIMiTER ;
Second one is:
DELIMITER $$
DROP FUNCTION IF EXISTS fp_splitadjprice;
CREATE FUNCTION fp_splitadjprice (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
DECLARE splitadjprice FLOAT;
DECLARE spinofffactor FLOAT;
SET splitfactor = 1.0;
SELECT fp_splitfactor(id, startdate) INTO splitfactor;
SELECT (p_price * splitfactor) INTO splitadjprice
FROM fp_v2_fp_basic_prices
WHERE fsym_id = id AND p_date = startdate;
RETURN splitadjprice;
END$$
DELIMITER ;
I then try to exectute a query as the following:
SELECT
p.fsym_id,
b.p_co_sec_name_desc AS Company_Name,
b.region AS Region,
p.p_date,
p.p_price AS Unadjusted_Price,
fp_splitadjprice(p.fsym_id,p_date) AS Adjusted_Price
FROM
fp_v2_fp_basic_prices p
LEFT JOIN (
SELECT r2.region, b2.p_co_sec_name_desc, b2.fsym_id
FROM fp_v2_fp_sec_coverage b2
LEFT JOIN sym_v1_sym_region r2 ON b2.fsym_id = r2.fsym_id
WHERE r2.region = "EUR") b
ON b.fsym_id =p.fsym_id
So basically my query calls the second function, which then calls the first function in order to return a value to the query. The execution is extremely slow though, but I do not understand why that is the case?
mysql mysql-workbench
mysql mysql-workbench
edited Nov 14 '18 at 16:43
Chris
asked Nov 14 '18 at 16:36
ChrisChris
1189
1189
your query call recursively other query .. this could be the reason for the slow execution .. looking to the code .. could be that you can get your result using only query an not function (that recursively perform query ) .. you should add a proper (simple) data sample and the expected resultt ..
– scaisEdge
Nov 14 '18 at 17:23
@scaisEdge hmm, I will try to see if I can add a dataset. But I can't be the first one who has called functions from functions in MySQL before I reckon?
– Chris
Nov 14 '18 at 17:50
Yes you are not the firts .. nested function are normally used but if for each row you select you invoke a function that perform select .. and so on .. the performance can't be good..In SQL an approach based on the Set Theory is usually better that an algortmic approach ..
– scaisEdge
Nov 14 '18 at 17:53
Well, when I perform a query calling only 1 function it is also extremely slow. This function has a datetime value though, but it is slow even when I limit the output to 10 rows. That does not make sense to at all?
– Chris
Nov 14 '18 at 18:03
so is only the query at the bottom of the question that is low .. ? ..
– scaisEdge
Nov 14 '18 at 18:48
|
show 1 more comment
your query call recursively other query .. this could be the reason for the slow execution .. looking to the code .. could be that you can get your result using only query an not function (that recursively perform query ) .. you should add a proper (simple) data sample and the expected resultt ..
– scaisEdge
Nov 14 '18 at 17:23
@scaisEdge hmm, I will try to see if I can add a dataset. But I can't be the first one who has called functions from functions in MySQL before I reckon?
– Chris
Nov 14 '18 at 17:50
Yes you are not the firts .. nested function are normally used but if for each row you select you invoke a function that perform select .. and so on .. the performance can't be good..In SQL an approach based on the Set Theory is usually better that an algortmic approach ..
– scaisEdge
Nov 14 '18 at 17:53
Well, when I perform a query calling only 1 function it is also extremely slow. This function has a datetime value though, but it is slow even when I limit the output to 10 rows. That does not make sense to at all?
– Chris
Nov 14 '18 at 18:03
so is only the query at the bottom of the question that is low .. ? ..
– scaisEdge
Nov 14 '18 at 18:48
your query call recursively other query .. this could be the reason for the slow execution .. looking to the code .. could be that you can get your result using only query an not function (that recursively perform query ) .. you should add a proper (simple) data sample and the expected resultt ..
– scaisEdge
Nov 14 '18 at 17:23
your query call recursively other query .. this could be the reason for the slow execution .. looking to the code .. could be that you can get your result using only query an not function (that recursively perform query ) .. you should add a proper (simple) data sample and the expected resultt ..
– scaisEdge
Nov 14 '18 at 17:23
@scaisEdge hmm, I will try to see if I can add a dataset. But I can't be the first one who has called functions from functions in MySQL before I reckon?
– Chris
Nov 14 '18 at 17:50
@scaisEdge hmm, I will try to see if I can add a dataset. But I can't be the first one who has called functions from functions in MySQL before I reckon?
– Chris
Nov 14 '18 at 17:50
Yes you are not the firts .. nested function are normally used but if for each row you select you invoke a function that perform select .. and so on .. the performance can't be good..In SQL an approach based on the Set Theory is usually better that an algortmic approach ..
– scaisEdge
Nov 14 '18 at 17:53
Yes you are not the firts .. nested function are normally used but if for each row you select you invoke a function that perform select .. and so on .. the performance can't be good..In SQL an approach based on the Set Theory is usually better that an algortmic approach ..
– scaisEdge
Nov 14 '18 at 17:53
Well, when I perform a query calling only 1 function it is also extremely slow. This function has a datetime value though, but it is slow even when I limit the output to 10 rows. That does not make sense to at all?
– Chris
Nov 14 '18 at 18:03
Well, when I perform a query calling only 1 function it is also extremely slow. This function has a datetime value though, but it is slow even when I limit the output to 10 rows. That does not make sense to at all?
– Chris
Nov 14 '18 at 18:03
so is only the query at the bottom of the question that is low .. ? ..
– scaisEdge
Nov 14 '18 at 18:48
so is only the query at the bottom of the question that is low .. ? ..
– scaisEdge
Nov 14 '18 at 18:48
|
show 1 more comment
1 Answer
1
active
oldest
votes
I found out that the slow execution was entire due to MySQL workbench not handling large datasets well. Once I migrated everything to BigQuery on Google Cloud everything worked perfectly.
STAY AWAY FROM CALLING FUNCTIONS ON LARGE DATASETS IN MySQL Workbench!
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53304894%2fvery-slow-execution-when-calling-function%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I found out that the slow execution was entire due to MySQL workbench not handling large datasets well. Once I migrated everything to BigQuery on Google Cloud everything worked perfectly.
STAY AWAY FROM CALLING FUNCTIONS ON LARGE DATASETS IN MySQL Workbench!
add a comment |
I found out that the slow execution was entire due to MySQL workbench not handling large datasets well. Once I migrated everything to BigQuery on Google Cloud everything worked perfectly.
STAY AWAY FROM CALLING FUNCTIONS ON LARGE DATASETS IN MySQL Workbench!
add a comment |
I found out that the slow execution was entire due to MySQL workbench not handling large datasets well. Once I migrated everything to BigQuery on Google Cloud everything worked perfectly.
STAY AWAY FROM CALLING FUNCTIONS ON LARGE DATASETS IN MySQL Workbench!
I found out that the slow execution was entire due to MySQL workbench not handling large datasets well. Once I migrated everything to BigQuery on Google Cloud everything worked perfectly.
STAY AWAY FROM CALLING FUNCTIONS ON LARGE DATASETS IN MySQL Workbench!
answered Nov 21 '18 at 20:10
ChrisChris
1189
1189
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53304894%2fvery-slow-execution-when-calling-function%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
your query call recursively other query .. this could be the reason for the slow execution .. looking to the code .. could be that you can get your result using only query an not function (that recursively perform query ) .. you should add a proper (simple) data sample and the expected resultt ..
– scaisEdge
Nov 14 '18 at 17:23
@scaisEdge hmm, I will try to see if I can add a dataset. But I can't be the first one who has called functions from functions in MySQL before I reckon?
– Chris
Nov 14 '18 at 17:50
Yes you are not the firts .. nested function are normally used but if for each row you select you invoke a function that perform select .. and so on .. the performance can't be good..In SQL an approach based on the Set Theory is usually better that an algortmic approach ..
– scaisEdge
Nov 14 '18 at 17:53
Well, when I perform a query calling only 1 function it is also extremely slow. This function has a datetime value though, but it is slow even when I limit the output to 10 rows. That does not make sense to at all?
– Chris
Nov 14 '18 at 18:03
so is only the query at the bottom of the question that is low .. ? ..
– scaisEdge
Nov 14 '18 at 18:48