Cryptography is a rather advanced subject, and it takes a bit of explanation. I'd recommend the books: The Art of Deception by Kevin Mitnik, and The Code Book by Simon Singh to actually figure out what public and private "keys" are.
My horrid explanation:
Keys are actually serialized matricies. The value inside a row and column of the matrix determines the vector operation that will be done. After doing enough of these value-determined operations from a pseudo-random matrix, it becomes impossible to generate the original matrix from the second. The first matrix that generates the second is the private key, and the public key is the second generated matrix.
Think of it as sending a message inside of a box, that's locked in another box. The public key unlocks one box, and the private key unlocks the other. The private key of the recipient is the only thing that can unlock the outer box (but that box can be locked-only with the corresponding public key), and the public key of the sender is the only thing that can unlock the inner box.