I thought .NET was supposed to make things easier, if anything, than unmanaged code.
-
I know this is a very special case but still i ran headlong into it. The easiest way to implement a B+ tree on disk is using a memory mapped file. I think this is what SQL Server does, but don't quote me. However, the only way you can access memory mapped files in C# is through .NET interop which makes it useless. Because one of the points of a memory mapped file is that you can do memory allocations that are backed by disk. There's no way in hell .NET can give you that in its current incarnation, even if one were to write a custom host, because of the way a GC system works. What I'd like var foo = new int[1000000]; // backed by disk, paged automatically What I'd have to do. somepointer = VirtualAlloc(...) Write(somepointer, data) etc etc basically it works like file i/o which defeats essentially the whole purpose. =(
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
I know this is a very special case but still i ran headlong into it. The easiest way to implement a B+ tree on disk is using a memory mapped file. I think this is what SQL Server does, but don't quote me. However, the only way you can access memory mapped files in C# is through .NET interop which makes it useless. Because one of the points of a memory mapped file is that you can do memory allocations that are backed by disk. There's no way in hell .NET can give you that in its current incarnation, even if one were to write a custom host, because of the way a GC system works. What I'd like var foo = new int[1000000]; // backed by disk, paged automatically What I'd have to do. somepointer = VirtualAlloc(...) Write(somepointer, data) etc etc basically it works like file i/o which defeats essentially the whole purpose. =(
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
C++/CLI
#SupportHeForShe Government can give you nothing but what it takes from somebody else. A government big enough to give you everything you want is big enough to take everything you've got, including your freedom.-Ezra Taft Benson You must accept 1 of 2 basic premises: Either we are alone in the universe or we are not alone. Either way, the implications are staggering!-Wernher von Braun
-
C++/CLI
#SupportHeForShe Government can give you nothing but what it takes from somebody else. A government big enough to give you everything you want is big enough to take everything you've got, including your freedom.-Ezra Taft Benson You must accept 1 of 2 basic premises: Either we are alone in the universe or we are not alone. Either way, the implications are staggering!-Wernher von Braun
yeah but, i really want to do this in all managed code. There's plenty of C++ code to do disk based B+ trees. plus i think those only run on windoze
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
yeah but, i really want to do this in all managed code. There's plenty of C++ code to do disk based B+ trees. plus i think those only run on windoze
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
C++/CLI is managed code. But you also have easy access to the Win32 api without pInvoke. :) It's true it only runs on win :zzz: for now. .Net 5.0 will resolve that. :)
#SupportHeForShe Government can give you nothing but what it takes from somebody else. A government big enough to give you everything you want is big enough to take everything you've got, including your freedom.-Ezra Taft Benson You must accept 1 of 2 basic premises: Either we are alone in the universe or we are not alone. Either way, the implications are staggering!-Wernher von Braun
-
C++/CLI is managed code. But you also have easy access to the Win32 api without pInvoke. :) It's true it only runs on win :zzz: for now. .Net 5.0 will resolve that. :)
#SupportHeForShe Government can give you nothing but what it takes from somebody else. A government big enough to give you everything you want is big enough to take everything you've got, including your freedom.-Ezra Taft Benson You must accept 1 of 2 basic premises: Either we are alone in the universe or we are not alone. Either way, the implications are staggering!-Wernher von Braun
it's mixed mode, as I understand it. Managed *and* unmanaged in the same PE. So it's not simply a .NET assembly but an assembly with a bag on the side of unmanaged code. I'm certain this was true when it was introduced. It probably still is. also i think there's still marshalling going on with managed->unmanaged calls in C++/CLI it might be more efficient than the standard marshalling though, but it's still marshalling.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
I know this is a very special case but still i ran headlong into it. The easiest way to implement a B+ tree on disk is using a memory mapped file. I think this is what SQL Server does, but don't quote me. However, the only way you can access memory mapped files in C# is through .NET interop which makes it useless. Because one of the points of a memory mapped file is that you can do memory allocations that are backed by disk. There's no way in hell .NET can give you that in its current incarnation, even if one were to write a custom host, because of the way a GC system works. What I'd like var foo = new int[1000000]; // backed by disk, paged automatically What I'd have to do. somepointer = VirtualAlloc(...) Write(somepointer, data) etc etc basically it works like file i/o which defeats essentially the whole purpose. =(
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
So write an unmanaged assembly with C++ to do the heavy b-tree lifting...
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013 -
So write an unmanaged assembly with C++ to do the heavy b-tree lifting...
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013There's already a ton of B+ trees in C++ I was going for a managed code version that would run on any .NET capable platform. I have one that's in memory. I just need to make it diskable, but it's easier said than done.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
I know this is a very special case but still i ran headlong into it. The easiest way to implement a B+ tree on disk is using a memory mapped file. I think this is what SQL Server does, but don't quote me. However, the only way you can access memory mapped files in C# is through .NET interop which makes it useless. Because one of the points of a memory mapped file is that you can do memory allocations that are backed by disk. There's no way in hell .NET can give you that in its current incarnation, even if one were to write a custom host, because of the way a GC system works. What I'd like var foo = new int[1000000]; // backed by disk, paged automatically What I'd have to do. somepointer = VirtualAlloc(...) Write(somepointer, data) etc etc basically it works like file i/o which defeats essentially the whole purpose. =(
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
What's the problem? Just do exactly that:
int[] foo = new int[1000000];
And it'll be allocated for you from the LOH. Heck, you can do this if you want to:
int[] foo = new int[500000000];
Provided the max index fits in 31 bits (and the whole item is less than 2GB) .NET will let you have it if it can.
Sent from my Amstrad PC 1640 Never throw anything away, Griff Bad command or file name. Bad, bad command! Sit! Stay! Staaaay... AntiTwitter: @DalekDave is now a follower!
-
it's mixed mode, as I understand it. Managed *and* unmanaged in the same PE. So it's not simply a .NET assembly but an assembly with a bag on the side of unmanaged code. I'm certain this was true when it was introduced. It probably still is. also i think there's still marshalling going on with managed->unmanaged calls in C++/CLI it might be more efficient than the standard marshalling though, but it's still marshalling.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
Yes, you understand correctly. As for marshaling, it depends on what part of the code is doing the the call. If standard C++, then no. If managed, depends on the types involved.
#SupportHeForShe Government can give you nothing but what it takes from somebody else. A government big enough to give you everything you want is big enough to take everything you've got, including your freedom.-Ezra Taft Benson You must accept 1 of 2 basic premises: Either we are alone in the universe or we are not alone. Either way, the implications are staggering!-Wernher von Braun
-
There's already a ton of B+ trees in C++ I was going for a managed code version that would run on any .NET capable platform. I have one that's in memory. I just need to make it diskable, but it's easier said than done.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
You do know about the
System.IO.MemoryMappedFilesnamespace, right? MemoryMappedFile Class (System.IO.MemoryMappedFiles) | Microsoft Docs[^] If you have a C++ version of your app, you could run them side-by-side and compare performance.".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013 -
You do know about the
System.IO.MemoryMappedFilesnamespace, right? MemoryMappedFile Class (System.IO.MemoryMappedFiles) | Microsoft Docs[^] If you have a C++ version of your app, you could run them side-by-side and compare performance.".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013Yes, though it was added to .NET after my initial attempt at using mem mapped files from C#. Besides all that is is a wrapper like the one i had written years ago. It doesn't change the basic problem which is: var foo = new int[1000000]; //backed by disk, paged automatically, in C/C++ it's mainly because you can't use pointers in C#, and even if you use unsafe, you cannot pin objects to specific addresses in memory
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
What's the problem? Just do exactly that:
int[] foo = new int[1000000];
And it'll be allocated for you from the LOH. Heck, you can do this if you want to:
int[] foo = new int[500000000];
Provided the max index fits in 31 bits (and the whole item is less than 2GB) .NET will let you have it if it can.
Sent from my Amstrad PC 1640 Never throw anything away, Griff Bad command or file name. Bad, bad command! Sit! Stay! Staaaay... AntiTwitter: @DalekDave is now a follower!
i know that, but it's not efficient and doesn't solve the basic issue, which is persistent, rapidly searchable storage.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
I know this is a very special case but still i ran headlong into it. The easiest way to implement a B+ tree on disk is using a memory mapped file. I think this is what SQL Server does, but don't quote me. However, the only way you can access memory mapped files in C# is through .NET interop which makes it useless. Because one of the points of a memory mapped file is that you can do memory allocations that are backed by disk. There's no way in hell .NET can give you that in its current incarnation, even if one were to write a custom host, because of the way a GC system works. What I'd like var foo = new int[1000000]; // backed by disk, paged automatically What I'd have to do. somepointer = VirtualAlloc(...) Write(somepointer, data) etc etc basically it works like file i/o which defeats essentially the whole purpose. =(
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
Quote:
because of the way a GC system works
:-D :-D :-D[^].
Oh boy, you've been paying attention GC is great for string management compared to a traditional heap. for mapping a b-tree or b+tree to disk using memory mapped files not so much. It would be cool if .NET had a mechanism whereby you could create uncollected heaps that you manually destroy, and could allocate objects to them somehow. Maybe by making an appdomain with an UN GC'd heap in it or something. I know you can suspend garbage collection but that's not really what i'd be after because that impacts all objects.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
I know this is a very special case but still i ran headlong into it. The easiest way to implement a B+ tree on disk is using a memory mapped file. I think this is what SQL Server does, but don't quote me. However, the only way you can access memory mapped files in C# is through .NET interop which makes it useless. Because one of the points of a memory mapped file is that you can do memory allocations that are backed by disk. There's no way in hell .NET can give you that in its current incarnation, even if one were to write a custom host, because of the way a GC system works. What I'd like var foo = new int[1000000]; // backed by disk, paged automatically What I'd have to do. somepointer = VirtualAlloc(...) Write(somepointer, data) etc etc basically it works like file i/o which defeats essentially the whole purpose. =(
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
honey the codewitch wrote:
the only way you can access memory mapped files in C# is through .NET interop
Really? :-D Memory-Mapped Files | Microsoft Docs[^] MemoryMappedFile Class (System.IO.MemoryMappedFiles) | Microsoft Docs[^] (Added in .NET 4.0)
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
-
honey the codewitch wrote:
the only way you can access memory mapped files in C# is through .NET interop
Really? :-D Memory-Mapped Files | Microsoft Docs[^] MemoryMappedFile Class (System.IO.MemoryMappedFiles) | Microsoft Docs[^] (Added in .NET 4.0)
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
The MMF class is just a wrapper around the Interop code.
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013 -
The MMF class is just a wrapper around the Interop code.
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013As is quite a lot of the BCL. :)
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
-
honey the codewitch wrote:
the only way you can access memory mapped files in C# is through .NET interop
Really? :-D Memory-Mapped Files | Microsoft Docs[^] MemoryMappedFile Class (System.IO.MemoryMappedFiles) | Microsoft Docs[^] (Added in .NET 4.0)
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
yeah i forgot about it, but it doesn't change the same underlying issue, and isn't useful to me. It's the same thing as the wrapper i wrote a decade or more or so ago.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
C++/CLI
#SupportHeForShe Government can give you nothing but what it takes from somebody else. A government big enough to give you everything you want is big enough to take everything you've got, including your freedom.-Ezra Taft Benson You must accept 1 of 2 basic premises: Either we are alone in the universe or we are not alone. Either way, the implications are staggering!-Wernher von Braun
That Ezra Taft Benson, while true, is one of the dumbest remarks I've seen, and is used by some groups to put down government. The point of government is to take your money and do what's best for the people as a whole. You can argue that isn't what happens in some cases, but that misses the point. Government generally is the only entity that takes risks when the reward isn't obvious. That where just about all of our technology comes from because industry won't invest without the promise of fairly quick rewards.
-
That Ezra Taft Benson, while true, is one of the dumbest remarks I've seen, and is used by some groups to put down government. The point of government is to take your money and do what's best for the people as a whole. You can argue that isn't what happens in some cases, but that misses the point. Government generally is the only entity that takes risks when the reward isn't obvious. That where just about all of our technology comes from because industry won't invest without the promise of fairly quick rewards.
I could write pages on this but I fear aside from not being read, it would wax far too political for the lounge. I was anarchist for 26 years. I didn't so much leave it behind as I evolved, just so you have some idea of where I'm coming from. I've never particularly been a fan of states. That having been said, you're essentially right, with the following caveats: Governments are complex adaptive systems. In lay terms, they consist of so many people that they take on a life of their own. The actions of the whole are not necessarily reflective of its individual agents. The problem with this is such systems either grow or they wither and die. Stasis is rare. And they defend themselves against attempts to undermine them. Positive** prospects like limiting government do not limit government, they create defacto private government with state power and no accountability. ** i'm not using the term like "good" - i'm using it as in active vs. passive That gets dangerous and ugly when the continuation of the system is more important to the system than acting out the will of the people it's supposed to represent. And all of them fall down under the right or rather, wrong conditions. No system can perfectly contain the potential for revolt. So with those limitations in mind, I think it's a good idea to be wary of government, but I think most people regard it as necessary because anarchism doesn't scale. :)
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
