0

I want to buil a sequential LSTM model that predicts binary classification at every time step. More exactly, I want to predict an output for every paragraph in my texts (48 is the number of paragraphs). This is my code:

shape = np.shape(train_x) # 3D: number of texts, number of padded paragraphs, number of features
n = shape[0]  # number of texts
time_steps = shape[1]  # number of padded pars
features = shape[2]  # number of features

model = Sequential()
model.add(layers.Masking(mask_value=0.0, input_shape=(time_steps, features)))
model.add(layers.LSTM(128, return_sequences=True, return_state=False))  
model.add(layers.TimeDistributed(layers.Dense(1)))
model.compile(loss='categorical_crossentropy', optimizer='adam')
model.summary()

#train_x = np.array(train_x).reshape(2, input_shape, 3)
train_x = tf.convert_to_tensor(train_x)  # data needs to be tensor object
train_y = tf.convert_to_tensor(train_y)
model.fit(train_x, train_y, batch_size=2)

predictions = model.predict(test_x)

This is the error message I get:

ValueError: Can not squeeze dim[1], expected a dimension of 1, 
got 48 for '{{node categorical_crossentropy/weighted_loss/Squeeze}} = Squeeze[T=DT_FLOAT, 
squeeze_dims=[-1]](Cast)' with input shapes: [2,48].

I don't really know what to do with this, do I need to reshape my data? How? Or do I need to change something in the model? Thanks!

This is the entire traceback:

Traceback (most recent call last):
  File "program.py", line 247, in <module>
    eval_scores = train_classifier(x_train, y_train_sc, x_test, y_test_sc)
  File "program.py", line 201, in train_classifier
    model.fit(train_x, train_y, batch_size=2)
  File "C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py", line 108, in _method_wrapper
    return method(self, *args, **kwargs)
  File "C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1098, in fit
    tmp_logs = train_function(iterator)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\def_function.py", line 780, in __call__
    result = self._call(*args, **kwds)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\def_function.py", line 823, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\def_function.py", line 696, in _initialize
    self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\function.py", line 2855, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\function.py", line 3213, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\function.py", line 3065, in _create_graph_function
    func_graph_module.func_graph_from_py_func(
  File "C:\Python38\lib\site-packages\tensorflow\python\framework\func_graph.py", line 986, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\def_function.py", line 600, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "C:\Python38\lib\site-packages\tensorflow\python\framework\func_graph.py", line 973, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

    C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py:806 train_function  *
        return step_function(self, iterator)
    C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py:796 step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    C:\Python38\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:1211 run
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    C:\Python38\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2585 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    C:\Python38\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2945 _call_for_each_replica
        return fn(*args, **kwargs)
    C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py:789 run_step  **
        outputs = model.train_step(data)
    C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py:748 train_step
        loss = self.compiled_loss(
    C:\Python38\lib\site-packages\tensorflow\python\keras\engine\compile_utils.py:204 __call__
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    C:\Python38\lib\site-packages\tensorflow\python\keras\losses.py:150 __call__
        return losses_utils.compute_weighted_loss(
    C:\Python38\lib\site-packages\tensorflow\python\keras\utils\losses_utils.py:111 compute_weighted_loss
        weighted_losses = tf_losses_utils.scale_losses_by_sample_weight(
    C:\Python38\lib\site-packages\tensorflow\python\ops\losses\util.py:142 scale_losses_by_sample_weight
        losses, _, sample_weight = squeeze_or_expand_dimensions(
    C:\Python38\lib\site-packages\tensorflow\python\ops\losses\util.py:95 squeeze_or_expand_dimensions
        sample_weight = array_ops.squeeze(sample_weight, [-1])
    C:\Python38\lib\site-packages\tensorflow\python\util\dispatch.py:201 wrapper
        return target(*args, **kwargs)
    C:\Python38\lib\site-packages\tensorflow\python\util\deprecation.py:507 new_func
        return func(*args, **kwargs)
    C:\Python38\lib\site-packages\tensorflow\python\ops\array_ops.py:4259 squeeze
        return gen_array_ops.squeeze(input, axis, name)
    C:\Python38\lib\site-packages\tensorflow\python\ops\gen_array_ops.py:10043 squeeze
        _, _, _op, _outputs = _op_def_library._apply_op_helper(
    C:\Python38\lib\site-packages\tensorflow\python\framework\op_def_library.py:742 _apply_op_helper
        op = g._create_op_internal(op_type_name, inputs, dtypes=None,
    C:\Python38\lib\site-packages\tensorflow\python\framework\func_graph.py:591 _create_op_internal
        return super(FuncGraph, self)._create_op_internal(  # pylint: disable=protected-access
    C:\Python38\lib\site-packages\tensorflow\python\framework\ops.py:3477 _create_op_internal
        ret = Operation(
    C:\Python38\lib\site-packages\tensorflow\python\framework\ops.py:1974 __init__
        self._c_op = _create_c_op(self._graph, node_def, inputs,
    C:\Python38\lib\site-packages\tensorflow\python\framework\ops.py:1815 _create_c_op
        raise ValueError(str(e))

    ValueError: Can not squeeze dim[1], expected a dimension of 1, got 48 for '{{node categorical_crossentropy/weighted_loss/Squeeze}} = Squeeze[T=DT_FLOAT, squeeze_dims=[-1]](Cast)' with input shapes: [2,48].

I don't know if it has something to do with the batch size, I tried change that to time_steps, but then I get:

ValueError: Can not squeeze dim[1], expected a dimension of 1, got 48 for '{{node categorical_crossentropy/weighted_loss/Squeeze}} = Squeeze[T=DT_FLOAT, squeeze_dims=[-1]](Cast)' with input shapes: [?,48].

I also tried to change it to 1:

    ValueError: Can not squeeze dim[1], expected a dimension of 1, got 48 for '{{node categorical_crossentropy/weighted_loss/Squeeze}} = Squeeze[T=DT_FLOAT, squeeze_dims=[-1]](Cast)' with input shapes: [1,48].
  • 3
    This seems like more of a Python question than a linguistics question… – Janus Bahs Jacquet Nov 21 at 15:51
  • 1
    Thank you for your input. However, it's a computational linguistics questions, so it's about linguistic research. And as "Linguistics Stack Exchange is a question and answer site for professional linguists and others with an interest in linguistic research and theory.", I think it should fit here – maeven Nov 21 at 16:06
  • Might be better to migrate to datascience.stackexchange.com – jick Nov 21 at 21:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.