CipherInputStream.read(byte[]) Dropping Final Block
-
I am writing test code that prints an encrypted string to an underlying ByteArrayOutputStream, then code to decrypt the bytes from that stream into another string. Here is my code:
byte[] keyBytes, ivBytes;
// Populate keyBytes and ivBytes with byte arrays of length 16 bytes (128 bits)final SecretKey key = new SecretKeySpec(keyBytes, "AES");
final IvParameterSpec iv = new IvParameterSpec(ivBytes);final Cipher encrypt = Cipher.getInstance("AES/CBC/PKCS5Padding");
final Cipher decrypt = Cipher.getInstance("AES/CBC/PKCS5Padding");encrypt.init(Cipher.ENCRYPT_MODE, key, iv);
decrypt.init(Cipher.DECRYPT_MODE, key, iv);final ByteArrayOutputStream outBytes = new ByteArrayOutputStream();
final CipherOutputStream out = new CipherOutputStream(outBytes, encrypt);out.write("This is a test, of course.".getBytes());
out.close();final byte[] encryptedBytes = outBytes.toByteArray();
final ByteArrayInputStream inBytes = new ByteArrayInputStream(encryptedBytes);
final CipherInputStream in = new CipherInputStream(inBytes, decrypt);final byte[] result = new byte[100];
final int byteCount = in.read(result);System.out.print('"');
System.out.print(new String(result, 0, byteCount));
System.out.println('"');For the above code, I expected the output to be "This is a test, of course.". However, I am getting the string, "This is a test, " (the final block is not being read). If I continue calling in.read(), I am able to get the remaining bytes out of the input stream, one byte at a time. This indicates (I think) that CipherInputStream is not simply catching and ignoring an invalid padding exception, which is what a few other posts I have found mentioned might be happening. When I run the above code, the following two properties hold: 1. encryptedBytes.length == 32, 2. new String(decrypt.doFinal(encryptedBytes)) creates a string with the expected value My question is, how in the heck to I get the CipherInputStream to read these bytes, without reading them one at a time? Is this expected behavior?
Sounds like somebody's got a case of the Mondays -Jeff
-
I am writing test code that prints an encrypted string to an underlying ByteArrayOutputStream, then code to decrypt the bytes from that stream into another string. Here is my code:
byte[] keyBytes, ivBytes;
// Populate keyBytes and ivBytes with byte arrays of length 16 bytes (128 bits)final SecretKey key = new SecretKeySpec(keyBytes, "AES");
final IvParameterSpec iv = new IvParameterSpec(ivBytes);final Cipher encrypt = Cipher.getInstance("AES/CBC/PKCS5Padding");
final Cipher decrypt = Cipher.getInstance("AES/CBC/PKCS5Padding");encrypt.init(Cipher.ENCRYPT_MODE, key, iv);
decrypt.init(Cipher.DECRYPT_MODE, key, iv);final ByteArrayOutputStream outBytes = new ByteArrayOutputStream();
final CipherOutputStream out = new CipherOutputStream(outBytes, encrypt);out.write("This is a test, of course.".getBytes());
out.close();final byte[] encryptedBytes = outBytes.toByteArray();
final ByteArrayInputStream inBytes = new ByteArrayInputStream(encryptedBytes);
final CipherInputStream in = new CipherInputStream(inBytes, decrypt);final byte[] result = new byte[100];
final int byteCount = in.read(result);System.out.print('"');
System.out.print(new String(result, 0, byteCount));
System.out.println('"');For the above code, I expected the output to be "This is a test, of course.". However, I am getting the string, "This is a test, " (the final block is not being read). If I continue calling in.read(), I am able to get the remaining bytes out of the input stream, one byte at a time. This indicates (I think) that CipherInputStream is not simply catching and ignoring an invalid padding exception, which is what a few other posts I have found mentioned might be happening. When I run the above code, the following two properties hold: 1. encryptedBytes.length == 32, 2. new String(decrypt.doFinal(encryptedBytes)) creates a string with the expected value My question is, how in the heck to I get the CipherInputStream to read these bytes, without reading them one at a time? Is this expected behavior?
Sounds like somebody's got a case of the Mondays -Jeff
If your application is something like the one shown in the example, like encrypting and decrypting static (previously known, as opposed to a stream) bytes, then use Cipher class instead, it's easier
-
I am writing test code that prints an encrypted string to an underlying ByteArrayOutputStream, then code to decrypt the bytes from that stream into another string. Here is my code:
byte[] keyBytes, ivBytes;
// Populate keyBytes and ivBytes with byte arrays of length 16 bytes (128 bits)final SecretKey key = new SecretKeySpec(keyBytes, "AES");
final IvParameterSpec iv = new IvParameterSpec(ivBytes);final Cipher encrypt = Cipher.getInstance("AES/CBC/PKCS5Padding");
final Cipher decrypt = Cipher.getInstance("AES/CBC/PKCS5Padding");encrypt.init(Cipher.ENCRYPT_MODE, key, iv);
decrypt.init(Cipher.DECRYPT_MODE, key, iv);final ByteArrayOutputStream outBytes = new ByteArrayOutputStream();
final CipherOutputStream out = new CipherOutputStream(outBytes, encrypt);out.write("This is a test, of course.".getBytes());
out.close();final byte[] encryptedBytes = outBytes.toByteArray();
final ByteArrayInputStream inBytes = new ByteArrayInputStream(encryptedBytes);
final CipherInputStream in = new CipherInputStream(inBytes, decrypt);final byte[] result = new byte[100];
final int byteCount = in.read(result);System.out.print('"');
System.out.print(new String(result, 0, byteCount));
System.out.println('"');For the above code, I expected the output to be "This is a test, of course.". However, I am getting the string, "This is a test, " (the final block is not being read). If I continue calling in.read(), I am able to get the remaining bytes out of the input stream, one byte at a time. This indicates (I think) that CipherInputStream is not simply catching and ignoring an invalid padding exception, which is what a few other posts I have found mentioned might be happening. When I run the above code, the following two properties hold: 1. encryptedBytes.length == 32, 2. new String(decrypt.doFinal(encryptedBytes)) creates a string with the expected value My question is, how in the heck to I get the CipherInputStream to read these bytes, without reading them one at a time? Is this expected behavior?
Sounds like somebody's got a case of the Mondays -Jeff
Skippums wrote:
1. encryptedBytes.length == 32,
In PKCS5 padding used here, the block length is fixed( may be 16), since the bytes available are insufficient, the call is blocked until the full 32 bytes arrive the stream or end of stream is detected, that is -1 is returned from the parent stream (ByteArrayInputStream in this case). A cryptanalyst will be able to explain you better about the working of the padding schemes and ciphers.
-
Skippums wrote:
1. encryptedBytes.length == 32,
In PKCS5 padding used here, the block length is fixed( may be 16), since the bytes available are insufficient, the call is blocked until the full 32 bytes arrive the stream or end of stream is detected, that is -1 is returned from the parent stream (ByteArrayInputStream in this case). A cryptanalyst will be able to explain you better about the working of the padding schemes and ciphers.
Sorry, I should have clarified in my post; I know that the length of the encrypted byte array was correct at 32 bytes long. When I searched for an answer to my question, the first thing people trying to help ask is, "what is the length of your encrypted stream?" Most people were getting something that was NOT a multiple of the block size, which indicates the error is earlier than the read phase. I posted that information to try and "prove" that the error was during the read phase, and NOT during the write phase (especially since I can get the information out of the stream if I use it in a unique way). As for your other answer, the code I posted was a minimal example of how to illustrate the problem, but in no way reflects how I am actually attempting to use the CipherInputStream/CipherOutputStream classes. Essentially, I am attempting to the interface to the crypto API significantly easier by creating a factory that returns a CipherInputStream/CipherOutputStream when the client passes in enumerated values for the cipher mode and padding mode. And yes, you are correct that for AES the block size is 16 bytes. For now, I just created a class called CipherInputStreamWrapper, and overloaded the read(byte[], int, int) method to iteratively call CipherInputStream.read(). This approach works, but I think it illustrates a flaw in the underlying implementation of CipherInputStream in Java 6 and 7 (tested in both). Thank you for taking the time to answer my post, any other ideas as to how to solve this problem are welcome!
Sounds like somebody's got a case of the Mondays -Jeff
-
I am writing test code that prints an encrypted string to an underlying ByteArrayOutputStream, then code to decrypt the bytes from that stream into another string. Here is my code:
byte[] keyBytes, ivBytes;
// Populate keyBytes and ivBytes with byte arrays of length 16 bytes (128 bits)final SecretKey key = new SecretKeySpec(keyBytes, "AES");
final IvParameterSpec iv = new IvParameterSpec(ivBytes);final Cipher encrypt = Cipher.getInstance("AES/CBC/PKCS5Padding");
final Cipher decrypt = Cipher.getInstance("AES/CBC/PKCS5Padding");encrypt.init(Cipher.ENCRYPT_MODE, key, iv);
decrypt.init(Cipher.DECRYPT_MODE, key, iv);final ByteArrayOutputStream outBytes = new ByteArrayOutputStream();
final CipherOutputStream out = new CipherOutputStream(outBytes, encrypt);out.write("This is a test, of course.".getBytes());
out.close();final byte[] encryptedBytes = outBytes.toByteArray();
final ByteArrayInputStream inBytes = new ByteArrayInputStream(encryptedBytes);
final CipherInputStream in = new CipherInputStream(inBytes, decrypt);final byte[] result = new byte[100];
final int byteCount = in.read(result);System.out.print('"');
System.out.print(new String(result, 0, byteCount));
System.out.println('"');For the above code, I expected the output to be "This is a test, of course.". However, I am getting the string, "This is a test, " (the final block is not being read). If I continue calling in.read(), I am able to get the remaining bytes out of the input stream, one byte at a time. This indicates (I think) that CipherInputStream is not simply catching and ignoring an invalid padding exception, which is what a few other posts I have found mentioned might be happening. When I run the above code, the following two properties hold: 1. encryptedBytes.length == 32, 2. new String(decrypt.doFinal(encryptedBytes)) creates a string with the expected value My question is, how in the heck to I get the CipherInputStream to read these bytes, without reading them one at a time? Is this expected behavior?
Sounds like somebody's got a case of the Mondays -Jeff
For anyone else looking for an answer to this, I couldn't find one. However, I am able to perform the following workaround: Instead of returning a CipherInputStream, I return a CipherInputStreamWrapper, which I define to be the following internal class:
class CipherInputStreamWrapper extends CipherInputStream {
CipherInputStreamWrapper(final InputStream is, final Cipher cipher) { super(is, cipher); } @Override public int read(final byte\[\] b) { return this.read(b, 0, b.length); } @Override public int read(final byte\[\] b, final int off, final int len) { for (int i = 0; i < len; ++i) { final int nextByte = this.read(); if (nextByte < 0) return i; b\[off + i\] = (byte)nextByte; } return len; }}
The implementation of read(byte[]) is actually identical to that in CipherInputStream in Java 6 and 7, but if the implentation changes in the future, NOT overloading that method would result in yet another erroneous implementation. I am always open to other ideas if anyone else has a cleaner solution!
Sounds like somebody's got a case of the Mondays -Jeff
-
For anyone else looking for an answer to this, I couldn't find one. However, I am able to perform the following workaround: Instead of returning a CipherInputStream, I return a CipherInputStreamWrapper, which I define to be the following internal class:
class CipherInputStreamWrapper extends CipherInputStream {
CipherInputStreamWrapper(final InputStream is, final Cipher cipher) { super(is, cipher); } @Override public int read(final byte\[\] b) { return this.read(b, 0, b.length); } @Override public int read(final byte\[\] b, final int off, final int len) { for (int i = 0; i < len; ++i) { final int nextByte = this.read(); if (nextByte < 0) return i; b\[off + i\] = (byte)nextByte; } return len; }}
The implementation of read(byte[]) is actually identical to that in CipherInputStream in Java 6 and 7, but if the implentation changes in the future, NOT overloading that method would result in yet another erroneous implementation. I am always open to other ideas if anyone else has a cleaner solution!
Sounds like somebody's got a case of the Mondays -Jeff
See javadoc for CipherInputStream[^]
This method blocks until input data is available or end of stream is reached or an exception is thrown.
Keep this in mind. The last block is not decrypted and returned, until end of stream is reached. Circumventing this approach MAY LEAD TO DATA CORRUPTION!
-
See javadoc for CipherInputStream[^]
This method blocks until input data is available or end of stream is reached or an exception is thrown.
Keep this in mind. The last block is not decrypted and returned, until end of stream is reached. Circumventing this approach MAY LEAD TO DATA CORRUPTION!
Sorry, I must be thick-headed, but I still don't understand. In my case, it seems (to me) that either: 1. Input data is available, or 2. The end of stream is reached I am still not getting how neither of these is true, regardless of whether it is the final block or not. Can you elaborate on what exactly is happening that is making both of those conditions false? Thanks,
Sounds like somebody's got a case of the Mondays -Jeff
-
Sorry, I must be thick-headed, but I still don't understand. In my case, it seems (to me) that either: 1. Input data is available, or 2. The end of stream is reached I am still not getting how neither of these is true, regardless of whether it is the final block or not. Can you elaborate on what exactly is happening that is making both of those conditions false? Thanks,
Sounds like somebody's got a case of the Mondays -Jeff
Lets Take an example of network streams, If socket is not closed and no data is being sent, then stream.available() will return 0, but end of stream is not reached, so the final block is not returned. Similar case can be with memory streams...
-
Lets Take an example of network streams, If socket is not closed and no data is being sent, then stream.available() will return 0, but end of stream is not reached, so the final block is not returned. Similar case can be with memory streams...
Ah, got it. I thought you were commenting on my specific example. So this comment was simply to caution me about specific cases that may break my proposed implementation, but does not explain why my example code in the original post is not returning the final block. Is this correct?
Sounds like somebody's got a case of the Mondays -Jeff
-
Ah, got it. I thought you were commenting on my specific example. So this comment was simply to caution me about specific cases that may break my proposed implementation, but does not explain why my example code in the original post is not returning the final block. Is this correct?
Sounds like somebody's got a case of the Mondays -Jeff
Skippums wrote:
So this comment was simply to caution me about specific cases that may break my proposed implementation
Yup, the comment was to caution you about the specific cases that may break your implementation. But your example might also be related. I never tried in on my system!
-
I am writing test code that prints an encrypted string to an underlying ByteArrayOutputStream, then code to decrypt the bytes from that stream into another string. Here is my code:
byte[] keyBytes, ivBytes;
// Populate keyBytes and ivBytes with byte arrays of length 16 bytes (128 bits)final SecretKey key = new SecretKeySpec(keyBytes, "AES");
final IvParameterSpec iv = new IvParameterSpec(ivBytes);final Cipher encrypt = Cipher.getInstance("AES/CBC/PKCS5Padding");
final Cipher decrypt = Cipher.getInstance("AES/CBC/PKCS5Padding");encrypt.init(Cipher.ENCRYPT_MODE, key, iv);
decrypt.init(Cipher.DECRYPT_MODE, key, iv);final ByteArrayOutputStream outBytes = new ByteArrayOutputStream();
final CipherOutputStream out = new CipherOutputStream(outBytes, encrypt);out.write("This is a test, of course.".getBytes());
out.close();final byte[] encryptedBytes = outBytes.toByteArray();
final ByteArrayInputStream inBytes = new ByteArrayInputStream(encryptedBytes);
final CipherInputStream in = new CipherInputStream(inBytes, decrypt);final byte[] result = new byte[100];
final int byteCount = in.read(result);System.out.print('"');
System.out.print(new String(result, 0, byteCount));
System.out.println('"');For the above code, I expected the output to be "This is a test, of course.". However, I am getting the string, "This is a test, " (the final block is not being read). If I continue calling in.read(), I am able to get the remaining bytes out of the input stream, one byte at a time. This indicates (I think) that CipherInputStream is not simply catching and ignoring an invalid padding exception, which is what a few other posts I have found mentioned might be happening. When I run the above code, the following two properties hold: 1. encryptedBytes.length == 32, 2. new String(decrypt.doFinal(encryptedBytes)) creates a string with the expected value My question is, how in the heck to I get the CipherInputStream to read these bytes, without reading them one at a time? Is this expected behavior?
Sounds like somebody's got a case of the Mondays -Jeff
If your program is something like the one proven in the example, like encrypting and decrypting fixed (previously known, in contrast to a stream) bytes, then use Cipher category instead, it's easier how to franchise your business