Very nice puzzle! A quick Python script that solves it as it's difficult to do with a pen and paper:

  #awesome feature from Python 3.9!
  from functools import cache
  
  @cache
  def ev(score, cards_left):
      if cards_left == 0: return 0
      up_cards_no = (cards_left - score) / 2
      p_up = up_cards_no / cards_left
      p_down = 1 - p_up
      return max(score, p_up * ev(score + 1, cards_left - 1) +  p_down * ev(score - 1, cards_left - 1))

  >>> ev(0,52)
  2.6244755489939253

Of course we should continue if the current score is lower than ev. Here is the script to find exact thresholds at which we should stop drawing more cards:

  def find_thresholds():
      print(f"EV of tha game: {ev(0,52)}") # fill the cache up
      for i in range (0,10):
          thres = 52 - i
          while ev(i, thres) > i:
              thres -= 2
          print(f"At score {i} you should stop drawing with {thres} cards left")

  >>> find_thresholds()
  EV of tha game: 2.6244755489939253
  At score 0 you should stop drawing with 0 cards left
  At score 1 you should stop drawing with 3 cards left
  At score 2 you should stop drawing with 8 cards left
  At score 3 you should stop drawing with 17 cards left
  At score 4 you should stop drawing with 28 cards left
  At score 5 you should stop drawing with 41 cards left
  At score 6 you should stop drawing with 46 cards left
  ...

Being up 6 or more we should always stop. The last interesting number is being up 5.