Educational: my buffer scroll C++ working code.
-
Hello. It my first message here, I'll try to do it the right way. The programming is my hobby (MinGW 32bit). About topic. I would like to represent my C++ code for buffer scroll. Perhaps I designing another bike, but still... What I mean about scrolling. For example, I have some buffer in which data starts somewhere in the middle:
char * buffer = "World!Hello, ";
But I need:
char * buffer = "Hello, World!";
So I need to cut out data at the end, move the rest of data to the end of buffer and paste cutted data to the beginning. This is "lossless buffer scrolling" of what I am talking about. This is a trivial job if ones use temporary buffer not less than half of the *buffer's length, few memcpy() statements and so on. But this is not my case. Condition 1. Source buffer size is very large (1/2 or 2/3 of total physical ram). So it is not good idea to allocate temporary buffer. Condition 2. I can not use data storage like HDD drive. I can not use any kind of swap techniques. (It will produces temporary system freeses.) Condition 3. I am not what to use virtual addressing techniques to virtually scroll buffer. Condition 4. The buffer size and scroll distance is arbitrary (sets as function parameters). So I created a scrolling buffer by permutation, which source code I would like to public. It is quite tested by brutte-force method (buffer size and scrolling distance were alternated, the result is compared with alternative scrolling techniques). In my opinion, the implementation of the method is unusual (or, at least, interesting). Here it is:
namespace CommonTools
{
/// Code by Shepel' Nikolay E., Moscow, Russia, published at 22 July 2017, http://www.codeproject.com.
#ifdef DEBUG
uint32_t scroll_raw_buffer_last_rounds = 0;
#endifvoid scroll_raw_buffer (void *data , const uint32_t buffer_size, uint32_t distance)
{
#ifdef DEBUG
scroll_raw_buffer_last_rounds = 0; // Statistics about rounds...
#endif
// Для реализации скроллинга необходим двойной запас по размеру буффера,
// т.е. buffer_size<(UINT_MAX/2) если использовать в функции 32х разрядные
// переменные, иначе можем выскачить за их максимальный размер, если
// надо будет буффер сместить влево на 1.// WARNING!!! // We need double reserve for addressing buffer elements (buffer\_size<(UINT\_MAX/2)). // Otherwise we can get register overflow while scrolling buffer with // size>=INT\_MAX to the -
Hello. It my first message here, I'll try to do it the right way. The programming is my hobby (MinGW 32bit). About topic. I would like to represent my C++ code for buffer scroll. Perhaps I designing another bike, but still... What I mean about scrolling. For example, I have some buffer in which data starts somewhere in the middle:
char * buffer = "World!Hello, ";
But I need:
char * buffer = "Hello, World!";
So I need to cut out data at the end, move the rest of data to the end of buffer and paste cutted data to the beginning. This is "lossless buffer scrolling" of what I am talking about. This is a trivial job if ones use temporary buffer not less than half of the *buffer's length, few memcpy() statements and so on. But this is not my case. Condition 1. Source buffer size is very large (1/2 or 2/3 of total physical ram). So it is not good idea to allocate temporary buffer. Condition 2. I can not use data storage like HDD drive. I can not use any kind of swap techniques. (It will produces temporary system freeses.) Condition 3. I am not what to use virtual addressing techniques to virtually scroll buffer. Condition 4. The buffer size and scroll distance is arbitrary (sets as function parameters). So I created a scrolling buffer by permutation, which source code I would like to public. It is quite tested by brutte-force method (buffer size and scrolling distance were alternated, the result is compared with alternative scrolling techniques). In my opinion, the implementation of the method is unusual (or, at least, interesting). Here it is:
namespace CommonTools
{
/// Code by Shepel' Nikolay E., Moscow, Russia, published at 22 July 2017, http://www.codeproject.com.
#ifdef DEBUG
uint32_t scroll_raw_buffer_last_rounds = 0;
#endifvoid scroll_raw_buffer (void *data , const uint32_t buffer_size, uint32_t distance)
{
#ifdef DEBUG
scroll_raw_buffer_last_rounds = 0; // Statistics about rounds...
#endif
// Для реализации скроллинга необходим двойной запас по размеру буффера,
// т.е. buffer_size<(UINT_MAX/2) если использовать в функции 32х разрядные
// переменные, иначе можем выскачить за их максимальный размер, если
// надо будет буффер сместить влево на 1.// WARNING!!! // We need double reserve for addressing buffer elements (buffer\_size<(UINT\_MAX/2)). // Otherwise we can get register overflow while scrolling buffer with // size>=INT\_MAX to theThis looks more like a Tip than a question and would be better posted at Submit a new Article[^]. But please read the posting guidelines[^] first.
-
Hello. It my first message here, I'll try to do it the right way. The programming is my hobby (MinGW 32bit). About topic. I would like to represent my C++ code for buffer scroll. Perhaps I designing another bike, but still... What I mean about scrolling. For example, I have some buffer in which data starts somewhere in the middle:
char * buffer = "World!Hello, ";
But I need:
char * buffer = "Hello, World!";
So I need to cut out data at the end, move the rest of data to the end of buffer and paste cutted data to the beginning. This is "lossless buffer scrolling" of what I am talking about. This is a trivial job if ones use temporary buffer not less than half of the *buffer's length, few memcpy() statements and so on. But this is not my case. Condition 1. Source buffer size is very large (1/2 or 2/3 of total physical ram). So it is not good idea to allocate temporary buffer. Condition 2. I can not use data storage like HDD drive. I can not use any kind of swap techniques. (It will produces temporary system freeses.) Condition 3. I am not what to use virtual addressing techniques to virtually scroll buffer. Condition 4. The buffer size and scroll distance is arbitrary (sets as function parameters). So I created a scrolling buffer by permutation, which source code I would like to public. It is quite tested by brutte-force method (buffer size and scrolling distance were alternated, the result is compared with alternative scrolling techniques). In my opinion, the implementation of the method is unusual (or, at least, interesting). Here it is:
namespace CommonTools
{
/// Code by Shepel' Nikolay E., Moscow, Russia, published at 22 July 2017, http://www.codeproject.com.
#ifdef DEBUG
uint32_t scroll_raw_buffer_last_rounds = 0;
#endifvoid scroll_raw_buffer (void *data , const uint32_t buffer_size, uint32_t distance)
{
#ifdef DEBUG
scroll_raw_buffer_last_rounds = 0; // Statistics about rounds...
#endif
// Для реализации скроллинга необходим двойной запас по размеру буффера,
// т.е. buffer_size<(UINT_MAX/2) если использовать в функции 32х разрядные
// переменные, иначе можем выскачить за их максимальный размер, если
// надо будет буффер сместить влево на 1.// WARNING!!! // We need double reserve for addressing buffer elements (buffer\_size<(UINT\_MAX/2)). // Otherwise we can get register overflow while scrolling buffer with // size>=INT\_MAX to theyes it belongs in tips but I also have a little tip for you. memmove from the C/C++ standards library can deal with source and destination memory overlapping and it can do it with no extra buffers at all. So you really only need the string search parts of your code and then use memmove to do the shuffle replacing the memcpy's memmove[^] memmove ensures that the original source bytes in the overlapping region are copied before being overwritten. It was probably a good learning excercise and you can rate how you went against a standards library and here is the code public source Implementation of memcopy[^] It's never wasted time to learn the standards libraries :-)
In vino veritas
-
yes it belongs in tips but I also have a little tip for you. memmove from the C/C++ standards library can deal with source and destination memory overlapping and it can do it with no extra buffers at all. So you really only need the string search parts of your code and then use memmove to do the shuffle replacing the memcpy's memmove[^] memmove ensures that the original source bytes in the overlapping region are copied before being overwritten. It was probably a good learning excercise and you can rate how you went against a standards library and here is the code public source Implementation of memcopy[^] It's never wasted time to learn the standards libraries :-)
In vino veritas
Quote:
memmove from the C/C++ standards library can deal with source and destination memory overlapping and it can do it with no extra buffers at all
I know it. But I can not implement memmove() to my task. memmove() will ovewrite some part of my buffer (or I have to use temporary buffer to store that part) so I'll loose some data. Perhaps there is a way to use it, but I do not yet know how.
Quote:
you really only need the string search parts of your code
char * buffer = "World!Hello, ";
It was only the small example to illustrate what I meaning under "buffer scrolling" term. In real code I have more that 100Mb binary (non-text) raw data which I have got from external device. I do not know (in advance) the size, and I do not know the move distance.
Echo7
-
This looks more like a Tip than a question and would be better posted at Submit a new Article[^]. But please read the posting guidelines[^] first.
Quote:
This looks more like a Tip than a question and would be better posted at Submit a new Article[^].
I'll do it, when I have read about rules.
Echo7