Evaluating GPT-4’s proficiency in addressing cryptography examinations

Figures & data

Table 1. Results in solving fundamental queries (oral exam).

Task field	Max pts	Points earned	Comments on errors
Basics of cryptology	8	8	–
Classical ciphers	22	20	Didn’t know bipartite cipher
Stat. measures in cryptography	18	16	Error in Friedman test equation
Cryptanalysis of classical ciphers	16	16	–
Machine ciphers	16	15	Enigma keyspace issue
Sec. objectives in modern crypto.	16	16	–
Symmetric modern ciphers	36	35	Stated that S-box is always 8 × 8
Crypto. hash functions	20	20	–
Asymmetric modern ciphers	14	14	–
Key exchange	12	12	–
Crypto. protocols	22	22	–
Quantum crypto.	8	8	–
Total	208	202 (97.11%)

Table 2. Results in solving a real crypto 1 exam.

Task field	Max pts	Points earned	Comments or errors
Basics	10	9.5	Missed certificate transmission step in TLS
Entropy	12	9	Incorrect password entropy formula
LFSRs and KSGs	20	20	–
Block Ciphers	16	12.5	Used undecryptable scheme for decryption
RSA	13	10.5	Several computation errors
Hash Functions	19	19	–
Total	90	80.5 (89.44%)

Table 3. Results in solving advanced crypto exercises.

#	Task	Pts	Solved	Got	Comments
Crypto mathematics
1.1	Prove ineligibility of functions	9	100%	9	Tasks likely used in training.
Stream ciphers
2.1.a	Guess and determine on Geffe	10	95%	9.5	Good but simpler approach exists.
2.1.b	Evaluate output biases	10	100%	10
2.1.c	Attacking modified Geffe generator	10	100%	10
2.1.d	Notice certain algebraic property	15	20%	3	Mistake at the beginning.
2.2	Algebraic attack on KSG	12	20%	2.4	Mistake at the beginning.
2.3	Compute LFSR output and period	7	60%	4.2	Correct output, but period not found.
2.4	Attack other KSGs	10	95%	9.5	Valid but inefficient.
2.5	Geffe generator attack	10	60%	6	Good algorithm, wrong computations.
2.6	Harder KSG attack	12	20%	2.4	Valid but off-task.
Block ciphers
3.1	Time to break AES 128	4	100%	4
3.2	Security of simplified AES	28	100%	28
Feistel networks
4.1	Pseudorandomness of Feistel networks	27	70%	18.9	False assumption for 1 out of 3 tasks.
4.2	Prove Feistel properties	10	100%	10
Differential cryptanalysis
5.1	Construct S-box differentials	12	80%	9.6	Correct start, later comp. errors.
Linear cryptanalysis
6.1	Prove linear S-box property	12	100%	12
Hash functions
7.2	Probability of flipping output by flipping inputs	9	60%	5.4	4/9 probabilities correct; issue with boolean function evaluations.
7.3	Evaluate hash function properties	24	66%	15.84	8/12 answers correct.
7.4	Hash function attribute implications	12	80%	9.6	False preimage resistance implication.
Provable security
8.1	Insecure RSA in IND-CPA	15	30%	4.5	Correct approach, misinterpreted task.
8.2	Properties of ElGamal scheme	18	100%	18
8.3	Perfect security conditions	24	66%	15.84	Misunderstood task.
8.4	Evaluate perfect secrecy schemes	15	67%	10.05	2/3 schemes evaluated correctly.
Public-key encryption schemes
9.1	Apply baby-step/giant-step algorithm	12	80%	9.6	Correct algorithm, incomplete computations.
9.2	Use ElGamal with parameters	9	90%	8.1	Arithmetic error.
9.3	Use RSA with parameters	9	70%	6.3	Correct formula, wrong computations.
9.4	Attack textbook RSA in practice	10	100%	10
9.5	Validate RSA decryption	18	100%	18
9.6	RSA vs Chinese remainder theorem	12	60%	7.2	Correct formulas, wrong computations.
In total		385	74.53%	286.93