Pfthb

I'm trying to train spacy NER, and I collected all my sentences, but many have embedded new line characters 'n' so when I enter the train data into my jupyter notebook, it failed with an error

 TRAIN_DATA = [('Who is 
 ^
SyntaxError: EOL when scanning string literal

What should I do with these?

Data looks like this (

TRAIN_DATA = [('Who is 
Shaka Khan?', 'entities': [(7, 17, 'PERSON')]),

Jupyter

If the problem is in jupyter you need to have 3x ' around strings that are on several lines like this

string=""" This string has many lines
 that continues here
 and here """

In your case that would be

TRAIN_DATA = [('''Who is 
 Shaka Khan?''', 'entities': [(7, 17, 'PERSON')])

Correct me if I'm wrong, but it looks like you've copy pasted the data, which is why this can happen. You could simply resolve the issue within Jupyter by just deleting the newline. Alternatively I would suggest that you import data to Jupyter not using copy paste.

Remove newline character

If you want to remove the newlines within string there are many options. Here is one

import re
string = re.sub('n', '', string)

Explanation

1. Line: import of regex modul


2. 
   

   Line: Use method 'sub' that
   substitutes first input 'n' with '' in string.

   
   
   

   out:
   ' This string has many lines that continues here and here '

   
   

Im guessing that you might be using pandas, so to do this on a column you can do the following:

df[col_name]=df[col_name].str.replace(r'^n','')