If Jesus Christ had actually existed as a historical person, we would expect Jewish and Pagan historians would have made some reference to him. Historicists claim there are such references, which I will detail below.
Suetonius, writing around 120 CE states:
"Since the Jews constantly made disturbances at the instigation of Chrestus, he [Claudius] expelled them from Rome." Life of Claudius (XXv.4)
Historicists claim that Chrestus is a misspelling for Christus, however Chrestus was actually a rather common name at the time and there's nothing to suggest it was a misspelling. After all, Suetonius correctly spelled Christians later in the book. Also, this passage implies that Chrestus was alive at the time, but it refers to the events of 49 CE, long after Christ was dead.
Tactitus wrote that:
"Consequently ... Nero fastened the guilt and inflicted the most exquisite tortures on a class hated for their abominations. Called Christians by the populace. Christus, from whom the name had its origin, suffered the extreme penalty during the reign of Tiberias at the hands of the Procurator Pontius Pilatus." Annals (XV.44)
Scholars believe this passage to be an interpolation because it isn't quoted before the 15th Century. If Tactitus had actually written this, certainly it would have been quoted by Christians earlier than this. Also, Pilate was a Prefect, not a Procurator, which Tactitus would have known if he was getting his information from Roman records. Even if this passage is genuine, it doesn't tell us that Jesus existed, Tactitus would just be repeating what he was told by Christians about their founder.
Pliny wrote a letter to the emperor Trajan around 100 CE saying:
"They also declared that the sum total of their guilt or error amounted to no more than this: that they had met regularly before dawn on a fixed day to chant verses alternately among themselves in honour of Christ as if to a god, and also to bind themselves by oath, not for any criminal purpose, but to abstain from theft, robbery, and adultery ..."
This passage, as well as others quoted by historicists, only proves that Christians existed, not that Christ did.
In the third century, Julius Africanus cites a passage written by Thallus that says earthquakes and darkness followed the death of Jesus. However, the original work by Thallus is lost, so this could have been an interpolation. Also, there is no other reference to earthquakes or darkness at the time from any other historian, which is quite unusual since such geological and meteorological events were dutifully recorded by historians back then.
There are countless later references. "The letter of Pontius Pilate which he wrote to the Roman Emperor, concerning our Lord Jesus Christ" is not thought to be authentic and most historians think it was actually written in the fifth century. In the Talmud, Jesus is said to be the illegitimate son of a Roman soldier, however, as it was written hundreds of years later, it's more likely a Jewish response to what Christians said of their founder.
Mythicists contend that in the early centuries of Christianity, the religion encompassed a wide range of movements. The Jesus of the Gospels is different from the Jesus of Hebrews or the Jesus of Paul's epistles. There were even some Christians who didn't believe Christ was crucified at all. The writings of these opposing sects were destroyed in the centuries of orthodoxy that followed or altered to be brought in line with the new opinion that Jesus had in fact existed. However, there is one such text that managed to survive.
Early Christian writer Minucius Felix wrote a dialogue between Caecilius, a pagan, and Octavius, a Christian. Caecilius accuses Christians of worshipping the head of an ass, reverencing the genitals of their priests, having orgies, and killing a baby and drinking its blood.
"And some say that the objects of their worship include a man who suffered death as a criminal, as well as the wretched wood of his cross; these are fitting altars for such depraved people, and they worship what they deserve."
Octavius is horrified by such accusations and counters that Christians don't do such things.
"Moreover, when you attribute to our religion the worship of a criminal and his cross, you wander far from the truth in thinking that a criminal deserved, or that a mortal man could be able, to be believed in as God. Miserable indeed is that man whose whole hope is dependent on a mortal, for such hope ceases with his death."
Minucius Felix is defending Christians from the wild accusations that they slaughter infants and drink their blood, have orgies, or worship someone who was crucified. Clearly the crucifixion story wasn't wide spread among all Christians in the early days.
We now turn from a Christian who didn't believe in the crucifixion to a Jew who supposedly did. The most famous historical references to Jesus are found in the Antiquities of the Jews written by the historian Josephus.
"So he [Ananus, son of Ananus the high priest] assembled the sanhedrin of judges, and brought before him the brother of Jesus, who was called Christ, whose name was James, and some others (or some of his companions) and when he had formed an accusation against them, he delivered them to be stoned." (Antiquities 20.9.1)
"The brother of Jesus, who was called Christ, whose name was James" is a rather awkward way of telling us someone's name. It is likely that the phrase "who was called Christ" was a marginal gloss which was later inserted into the text. "The brother of Jesus" doesn't necessarily refer to Jesus Christ, since Jesus was a rather common name at the time. Even if the phrase is a reference to Christ, "the brother of Jesus" could just as easily be a title as a familial identification.
Since Josephus spends much more time discussing other messiahs such as John the Baptist, it would be strange for him to make only a passing reference to Jesus without going into more detail, which brings us to the second passage. I'll include the text immediately preceding and following the reference to show it in context.
"But Pilate undertook to bring a current of water to Jerusalem, and did it with the sacred money, and derived the origin of the stream from the distance of two hundred furlongs. However, the Jews were not pleased with what had been done about this water; and many ten thousands of the people got together, and made a clamor against him, and insisted that he should leave off that design. [...] So he habited a great number of his soldiers in their habit, who carried daggers under their garments, and sent them to a place where they might surround them. So he bid the Jews himself go away; but they boldly casting reproaches upon him, he gave the soldiers that signal which had been beforehand agreed on; who laid upon them much greater blows than Pilate had commanded them, and equally punished those that were tumultuous, and those that were not; nor did they spare them in the least: and since the people were unarmed, and were caught by men prepared for what they were about, there were a great number of them slain by this means, and others of them ran away wounded. And thus an end was put to this sedition.
"Now there was about this time Jesus, a wise man, if it be lawful to call him a man; for he was a doer of wonderful works, a teacher of such men as receive the truth with pleasure. He drew over to him both many of the Jews and many of the Gentiles. He was [the] Christ. And when Pilate, at the suggestion of the principal men amongst us, had condemned him to the cross, those that loved him at the first did not forsake him; for he appeared to them alive again the third day; as the divine prophets had foretold these and ten thousand other wonderful things concerning him. And the tribe of Christians, so named from him, are not extinct at this day.
"About the same time also another sad calamity put the Jews into disorder, and certain shameful practices happened about the temple of Isis that was at Rome." (Antiquities 18)
The second paragraph above is called the Testimonium Flavianum. Even this expanded reference to Jesus is uncharacteristic of Josephus who usually goes into much more detail about even minor historical figures. Also, it doesn't make much sense for a Pharisaic Jew to declare that Jesus was the Christ.
What further makes the Testimonium suspect is the fact that it disrupts the flow of the passage. With the second paragraph removed, the narrative is much smoother. The first paragraph describes one calamity, the third paragraph refers to "another sad calamity." It seems as if the second paragraph were inserted by someone. Otherwise, we'd have to read the passage as saying that the "ten thousand other wonderful things" concerning Christ were, in fact, a calamity.
The case for interpolation is made stronger when we realize that no Christian writer refers to the Testimonium until two hundred years after it was written. Christian apologists such as Justin Martyr and Origen were undoubtedly familiar with Josephus, and would have jumped at the chance to answer their Pagan and Jewish critics with such a passage. Origen actually says that Josephus did not believe Jesus was the Christ.
The first person to cite this passage was Eusebius writing about 324 CE. Indeed, the vocabulary used in the Testimonium is more similar to Eusebius than to Josephus. For example, Josephus uses the Greek word poietes only to mean "poet", yet in the Testimonium it is used to mean "doer" (as part of the phrase "doer of wonderful works") Eusebius, on the other hand, does use poietes to mean "doer". The reference to Jesus as a "wise man" whose followers did not leave him after he was crucified is similar to the argument Eusebius makes in Demonstratio. In fact, Eusebius all but admits that he committed the forgery:
"Certainly the attestations I have already produced concerning our Savior may be sufficient. However, it may not be amiss, if, over and above, we make use of Josephus the Jew for a further witness." Demonstratio Evangelica, (Book III, pg. 124)
Eusebius is also our first source for Papias and Quadratus who both claim to have eyewitness testimony to the miracles of Jesus:
"If by chance anyone who had been in attendance on the elders should come my way, I inquired about the words of the elders — that is, what according to the elders Andrew or Peter said, or Philip, or Thomas or James, or John or Matthew or any other of the Lord’s disciples, and whatever Aristion and the elder John, the Lord’s disciples, were saying." Now lost document by Papias (according to Eusebius)
"The words of our Savior were always present, for they were true: those who were healed, those who rose from the dead, those who were not only seen in the act of being healed or raised, but were also always present, not merely when the Savior was living on earth, but also for a considerable time after his departure, so that some of them survived even to our own times." Now lost document by Quadratus (according to Eusebius)
So the question of whether there is proof of Jesus' existence outside the Gospels or not seems to depend on how reliable you consider Eusebuis of Caesarea to be. I can think of no better way to end this post than with a quote from him:
"It shall be legitimate and appropriate to use lies as a remedy." Eusebius of Caesarea, Preparation for the Gospel, 12:31