Specialize std::copy for custom iterators



.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;








2















From what I understand one can write template specializations for function template in the std namespace. I have written a CircularBuffer<T> class and implemented a random access iterator for this class. As is the std::copy() algorithm works with my custom iterators but it is not optimal. It iterates through the range and copies elements one by one. I could write a more optimal implementation for trivially copiable types by using std::memcpy() on internal pointers.



My question is that even possible? I know how I would create an overload of std::copy() as a template with output iterator being the template parameter. But that cannot be done since you can only write template specializations of function templates in the std namespace. Any help in pointing me in the right direction would be helpful. From what I could gather through Google, this can't be done, but I would love to be proven wrong :)










share|improve this question
























  • It seems that CircularBuffer is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespace std.

    – Jarod42
    Nov 15 '18 at 12:25






  • 2





    I wouldn't write any function specializations in new code.

    – StoryTeller
    Nov 15 '18 at 12:25












  • @StoryTeller it also states that it is fine as long as there are at least one program defined type involved.

    – darune
    Nov 15 '18 at 12:31






  • 1





    @darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.

    – StoryTeller
    Nov 15 '18 at 12:32






  • 1





    Just to clarify: your CircularBuffer::iterator is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar to std::deque?

    – Caleth
    Nov 15 '18 at 12:56

















2















From what I understand one can write template specializations for function template in the std namespace. I have written a CircularBuffer<T> class and implemented a random access iterator for this class. As is the std::copy() algorithm works with my custom iterators but it is not optimal. It iterates through the range and copies elements one by one. I could write a more optimal implementation for trivially copiable types by using std::memcpy() on internal pointers.



My question is that even possible? I know how I would create an overload of std::copy() as a template with output iterator being the template parameter. But that cannot be done since you can only write template specializations of function templates in the std namespace. Any help in pointing me in the right direction would be helpful. From what I could gather through Google, this can't be done, but I would love to be proven wrong :)










share|improve this question
























  • It seems that CircularBuffer is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespace std.

    – Jarod42
    Nov 15 '18 at 12:25






  • 2





    I wouldn't write any function specializations in new code.

    – StoryTeller
    Nov 15 '18 at 12:25












  • @StoryTeller it also states that it is fine as long as there are at least one program defined type involved.

    – darune
    Nov 15 '18 at 12:31






  • 1





    @darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.

    – StoryTeller
    Nov 15 '18 at 12:32






  • 1





    Just to clarify: your CircularBuffer::iterator is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar to std::deque?

    – Caleth
    Nov 15 '18 at 12:56













2












2








2


0






From what I understand one can write template specializations for function template in the std namespace. I have written a CircularBuffer<T> class and implemented a random access iterator for this class. As is the std::copy() algorithm works with my custom iterators but it is not optimal. It iterates through the range and copies elements one by one. I could write a more optimal implementation for trivially copiable types by using std::memcpy() on internal pointers.



My question is that even possible? I know how I would create an overload of std::copy() as a template with output iterator being the template parameter. But that cannot be done since you can only write template specializations of function templates in the std namespace. Any help in pointing me in the right direction would be helpful. From what I could gather through Google, this can't be done, but I would love to be proven wrong :)










share|improve this question
















From what I understand one can write template specializations for function template in the std namespace. I have written a CircularBuffer<T> class and implemented a random access iterator for this class. As is the std::copy() algorithm works with my custom iterators but it is not optimal. It iterates through the range and copies elements one by one. I could write a more optimal implementation for trivially copiable types by using std::memcpy() on internal pointers.



My question is that even possible? I know how I would create an overload of std::copy() as a template with output iterator being the template parameter. But that cannot be done since you can only write template specializations of function templates in the std namespace. Any help in pointing me in the right direction would be helpful. From what I could gather through Google, this can't be done, but I would love to be proven wrong :)







c++ iterator template-specialization






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 15 '18 at 12:30







rozina

















asked Nov 15 '18 at 12:22









rozinarozina

2,2201334




2,2201334












  • It seems that CircularBuffer is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespace std.

    – Jarod42
    Nov 15 '18 at 12:25






  • 2





    I wouldn't write any function specializations in new code.

    – StoryTeller
    Nov 15 '18 at 12:25












  • @StoryTeller it also states that it is fine as long as there are at least one program defined type involved.

    – darune
    Nov 15 '18 at 12:31






  • 1





    @darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.

    – StoryTeller
    Nov 15 '18 at 12:32






  • 1





    Just to clarify: your CircularBuffer::iterator is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar to std::deque?

    – Caleth
    Nov 15 '18 at 12:56

















  • It seems that CircularBuffer is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespace std.

    – Jarod42
    Nov 15 '18 at 12:25






  • 2





    I wouldn't write any function specializations in new code.

    – StoryTeller
    Nov 15 '18 at 12:25












  • @StoryTeller it also states that it is fine as long as there are at least one program defined type involved.

    – darune
    Nov 15 '18 at 12:31






  • 1





    @darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.

    – StoryTeller
    Nov 15 '18 at 12:32






  • 1





    Just to clarify: your CircularBuffer::iterator is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar to std::deque?

    – Caleth
    Nov 15 '18 at 12:56
















It seems that CircularBuffer is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespace std.

– Jarod42
Nov 15 '18 at 12:25





It seems that CircularBuffer is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespace std.

– Jarod42
Nov 15 '18 at 12:25




2




2





I wouldn't write any function specializations in new code.

– StoryTeller
Nov 15 '18 at 12:25






I wouldn't write any function specializations in new code.

– StoryTeller
Nov 15 '18 at 12:25














@StoryTeller it also states that it is fine as long as there are at least one program defined type involved.

– darune
Nov 15 '18 at 12:31





@StoryTeller it also states that it is fine as long as there are at least one program defined type involved.

– darune
Nov 15 '18 at 12:31




1




1





@darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.

– StoryTeller
Nov 15 '18 at 12:32





@darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.

– StoryTeller
Nov 15 '18 at 12:32




1




1





Just to clarify: your CircularBuffer::iterator is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar to std::deque?

– Caleth
Nov 15 '18 at 12:56





Just to clarify: your CircularBuffer::iterator is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar to std::deque?

– Caleth
Nov 15 '18 at 12:56












1 Answer
1






active

oldest

votes


















1














Maybe your question should be: should I do it ?



Users who make use of your class should already be aware of what std::copy is and how it works and inherently the performance implications. So providing a specialization could make matters worse. std::copy guarantees that N assignments are made; according to the standard:




Exactly (last - first) assignments




Also, it is not uncommon that when std::copy is used, it is also used with back_inserters or other manipulators, that will probably not play well with optimization.



However, you could, for instance, choose to provide a direct access to the buffer like in std::vector::data.



My question is that even possible?



One way around this issue that you seem to have is to export this knowledge - in a sense - to the users of your class. Just add an extra level of indirection. So instead of having iterators to the elements directly you will return iterators to the blocks of memory. Then you will be able to have Contiguous iterators.



https://en.cppreference.com/w/cpp/named_req/ContiguousIterator






share|improve this answer

























  • From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?

    – rozina
    Nov 15 '18 at 13:50












  • @rozina Since for ints, memcpy is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.

    – HolyBlackCat
    Nov 15 '18 at 14:08











  • I want to specialize std::copy for my CircularBuffer<int>. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization for std::copy.

    – rozina
    Nov 15 '18 at 14:11











  • @rozina It's based on the iterator properties as I see it. You have something more akin to a std::deque than a std::vector. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator

    – darune
    Nov 16 '18 at 12:12











  • I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data from CircularBuffer that uses a maximum of 2 calls of std::copy(), which for integers translates to std::memcpy(). I would love if std::copy() could somehow use my algorithm instead of its own. But since I am not supposed to overload std::copy(), my question is, if that is possible and how.

    – rozina
    Nov 16 '18 at 13:24












Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53319423%2fspecialize-stdcopy-for-custom-iterators%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









1














Maybe your question should be: should I do it ?



Users who make use of your class should already be aware of what std::copy is and how it works and inherently the performance implications. So providing a specialization could make matters worse. std::copy guarantees that N assignments are made; according to the standard:




Exactly (last - first) assignments




Also, it is not uncommon that when std::copy is used, it is also used with back_inserters or other manipulators, that will probably not play well with optimization.



However, you could, for instance, choose to provide a direct access to the buffer like in std::vector::data.



My question is that even possible?



One way around this issue that you seem to have is to export this knowledge - in a sense - to the users of your class. Just add an extra level of indirection. So instead of having iterators to the elements directly you will return iterators to the blocks of memory. Then you will be able to have Contiguous iterators.



https://en.cppreference.com/w/cpp/named_req/ContiguousIterator






share|improve this answer

























  • From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?

    – rozina
    Nov 15 '18 at 13:50












  • @rozina Since for ints, memcpy is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.

    – HolyBlackCat
    Nov 15 '18 at 14:08











  • I want to specialize std::copy for my CircularBuffer<int>. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization for std::copy.

    – rozina
    Nov 15 '18 at 14:11











  • @rozina It's based on the iterator properties as I see it. You have something more akin to a std::deque than a std::vector. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator

    – darune
    Nov 16 '18 at 12:12











  • I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data from CircularBuffer that uses a maximum of 2 calls of std::copy(), which for integers translates to std::memcpy(). I would love if std::copy() could somehow use my algorithm instead of its own. But since I am not supposed to overload std::copy(), my question is, if that is possible and how.

    – rozina
    Nov 16 '18 at 13:24
















1














Maybe your question should be: should I do it ?



Users who make use of your class should already be aware of what std::copy is and how it works and inherently the performance implications. So providing a specialization could make matters worse. std::copy guarantees that N assignments are made; according to the standard:




Exactly (last - first) assignments




Also, it is not uncommon that when std::copy is used, it is also used with back_inserters or other manipulators, that will probably not play well with optimization.



However, you could, for instance, choose to provide a direct access to the buffer like in std::vector::data.



My question is that even possible?



One way around this issue that you seem to have is to export this knowledge - in a sense - to the users of your class. Just add an extra level of indirection. So instead of having iterators to the elements directly you will return iterators to the blocks of memory. Then you will be able to have Contiguous iterators.



https://en.cppreference.com/w/cpp/named_req/ContiguousIterator






share|improve this answer

























  • From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?

    – rozina
    Nov 15 '18 at 13:50












  • @rozina Since for ints, memcpy is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.

    – HolyBlackCat
    Nov 15 '18 at 14:08











  • I want to specialize std::copy for my CircularBuffer<int>. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization for std::copy.

    – rozina
    Nov 15 '18 at 14:11











  • @rozina It's based on the iterator properties as I see it. You have something more akin to a std::deque than a std::vector. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator

    – darune
    Nov 16 '18 at 12:12











  • I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data from CircularBuffer that uses a maximum of 2 calls of std::copy(), which for integers translates to std::memcpy(). I would love if std::copy() could somehow use my algorithm instead of its own. But since I am not supposed to overload std::copy(), my question is, if that is possible and how.

    – rozina
    Nov 16 '18 at 13:24














1












1








1







Maybe your question should be: should I do it ?



Users who make use of your class should already be aware of what std::copy is and how it works and inherently the performance implications. So providing a specialization could make matters worse. std::copy guarantees that N assignments are made; according to the standard:




Exactly (last - first) assignments




Also, it is not uncommon that when std::copy is used, it is also used with back_inserters or other manipulators, that will probably not play well with optimization.



However, you could, for instance, choose to provide a direct access to the buffer like in std::vector::data.



My question is that even possible?



One way around this issue that you seem to have is to export this knowledge - in a sense - to the users of your class. Just add an extra level of indirection. So instead of having iterators to the elements directly you will return iterators to the blocks of memory. Then you will be able to have Contiguous iterators.



https://en.cppreference.com/w/cpp/named_req/ContiguousIterator






share|improve this answer















Maybe your question should be: should I do it ?



Users who make use of your class should already be aware of what std::copy is and how it works and inherently the performance implications. So providing a specialization could make matters worse. std::copy guarantees that N assignments are made; according to the standard:




Exactly (last - first) assignments




Also, it is not uncommon that when std::copy is used, it is also used with back_inserters or other manipulators, that will probably not play well with optimization.



However, you could, for instance, choose to provide a direct access to the buffer like in std::vector::data.



My question is that even possible?



One way around this issue that you seem to have is to export this knowledge - in a sense - to the users of your class. Just add an extra level of indirection. So instead of having iterators to the elements directly you will return iterators to the blocks of memory. Then you will be able to have Contiguous iterators.



https://en.cppreference.com/w/cpp/named_req/ContiguousIterator







share|improve this answer














share|improve this answer



share|improve this answer








edited Nov 16 '18 at 12:17

























answered Nov 15 '18 at 13:45









darunedarune

1,903820




1,903820












  • From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?

    – rozina
    Nov 15 '18 at 13:50












  • @rozina Since for ints, memcpy is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.

    – HolyBlackCat
    Nov 15 '18 at 14:08











  • I want to specialize std::copy for my CircularBuffer<int>. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization for std::copy.

    – rozina
    Nov 15 '18 at 14:11











  • @rozina It's based on the iterator properties as I see it. You have something more akin to a std::deque than a std::vector. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator

    – darune
    Nov 16 '18 at 12:12











  • I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data from CircularBuffer that uses a maximum of 2 calls of std::copy(), which for integers translates to std::memcpy(). I would love if std::copy() could somehow use my algorithm instead of its own. But since I am not supposed to overload std::copy(), my question is, if that is possible and how.

    – rozina
    Nov 16 '18 at 13:24


















  • From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?

    – rozina
    Nov 15 '18 at 13:50












  • @rozina Since for ints, memcpy is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.

    – HolyBlackCat
    Nov 15 '18 at 14:08











  • I want to specialize std::copy for my CircularBuffer<int>. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization for std::copy.

    – rozina
    Nov 15 '18 at 14:11











  • @rozina It's based on the iterator properties as I see it. You have something more akin to a std::deque than a std::vector. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator

    – darune
    Nov 16 '18 at 12:12











  • I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data from CircularBuffer that uses a maximum of 2 calls of std::copy(), which for integers translates to std::memcpy(). I would love if std::copy() could somehow use my algorithm instead of its own. But since I am not supposed to overload std::copy(), my question is, if that is possible and how.

    – rozina
    Nov 16 '18 at 13:24

















From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?

– rozina
Nov 15 '18 at 13:50






From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?

– rozina
Nov 15 '18 at 13:50














@rozina Since for ints, memcpy is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.

– HolyBlackCat
Nov 15 '18 at 14:08





@rozina Since for ints, memcpy is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.

– HolyBlackCat
Nov 15 '18 at 14:08













I want to specialize std::copy for my CircularBuffer<int>. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization for std::copy.

– rozina
Nov 15 '18 at 14:11





I want to specialize std::copy for my CircularBuffer<int>. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization for std::copy.

– rozina
Nov 15 '18 at 14:11













@rozina It's based on the iterator properties as I see it. You have something more akin to a std::deque than a std::vector. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator

– darune
Nov 16 '18 at 12:12





@rozina It's based on the iterator properties as I see it. You have something more akin to a std::deque than a std::vector. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator

– darune
Nov 16 '18 at 12:12













I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data from CircularBuffer that uses a maximum of 2 calls of std::copy(), which for integers translates to std::memcpy(). I would love if std::copy() could somehow use my algorithm instead of its own. But since I am not supposed to overload std::copy(), my question is, if that is possible and how.

– rozina
Nov 16 '18 at 13:24






I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data from CircularBuffer that uses a maximum of 2 calls of std::copy(), which for integers translates to std::memcpy(). I would love if std::copy() could somehow use my algorithm instead of its own. But since I am not supposed to overload std::copy(), my question is, if that is possible and how.

– rozina
Nov 16 '18 at 13:24




















draft saved

draft discarded
















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53319423%2fspecialize-stdcopy-for-custom-iterators%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Use pre created SQLite database for Android project in kotlin

Darth Vader #20

Ondo