re: using AI to learn to factor integers / find primes - probably not doable yet. There are neural networks that could model an algorithm that does it (memory networks, neural turing machine etc.). But any target algorithm would surely be too complicated for the neural net to converge towards it simply based on binary signals and gradient descent.

"The prime number theorem is equivalent to the statement that the nth prime number p_n satisfies

p_n ~ nlog(n)

the asymptotic notation meaning, again, that the relative error of this approximation approaches 0 as n increases without bound. For example, the 2^1017th prime number is 8512677386048191063, and (2^1017)log(2^1017) rounds to 7967418752291744388, a relative error of about 6.4%."

    from math import log
    def f(n, k):
      ln = log(n); lln = log(log(n))
      return ln + lln - 1 + (lln-2)/ln - ((lln**2) - 6*lln+k)/(2*ln*ln)

   >>> n = 2E17
   >>> lo = n * f(n, 11.847)
   >>> hi = n * f(n, 10.273)
   >>> lo
   8.512627944213742e+18
   >>> hi
   8.512727125430618e+18
   >>> exact = 8512677386048191063
   >>> (exact - lo)/exact
   5.80802398678381e-06
   >>> (exact - hi)/exact
   -5.842977499434444e-06

Look for Willans' formula (there's a nice explanation in the book "Hacker's Delight").